Bandwidth

From: ANT_THOMAS22 Nov 2012 11:51
To: ALL1 of 36
Is it normal for web hosts to count FTP uploads in the overall monthy bandwidth with hosting services?

For example, I have 5GB of bandwidth a month and I'm thinking of setting up an outside webcam with one of my RPis (no real point to this). I've currently got the webcam setup attached to another system just pointing out of the window to test the capture and upload script.

Currently it uploads every 20 seconds (might be too much) and in 8 days I've managed to use 1.3GB in bandwidth on just FTP. Obviously if I go to 1 minute per upload then that gets cut to 1/3 of the total.

I've also got a plan to upload multiple resolutions of images so different sizes are selectable, this is going to destroy the bandwidth.

Thinking about it, any form of data transfer should be counted because it's a service being used.

So I guess I need somewhere to store the files that has quite a large bandwidth allowance. Suggestions? (my internet connection is too poor to do it locally)
From: CHYRON (DSMITHHFX)22 Nov 2012 12:01
To: ANT_THOMAS 2 of 36
I'm pretty sure most do.
From: Matt22 Nov 2012 12:20
To: ANT_THOMAS 3 of 36
In my experience they do usually count FTP towards bandwidth.

How big are the files you're uploading, could they be optimised (reduce resolution and dimensions, different file format entirely?) and do you have compression enabled on the transfer?
From: ANT_THOMAS22 Nov 2012 12:34
To: Matt 4 of 36
They're JPEG
Currently the single 640x480 image is around 60KB - I do want to upload multiple resolutions though.
No idea about compression.

I've found a service I used in a past which is "free" and offers 100GB bandwidth. www.000webhost.com. It seems to be working fine there, but I remember them being a bit slow in the past, I'll see how it works out.
EDITED: 22 Nov 2012 17:56 by ANT_THOMAS
From: 99% of gargoyles look like (MR_BASTARD)22 Nov 2012 13:07
To: ANT_THOMAS 5 of 36
Why would a host not count FTP towards bandwidth? :?
From: ANT_THOMAS22 Nov 2012 13:11
To: 99% of gargoyles look like (MR_BASTARD) 6 of 36
I've realised that now. It was just a little shock to see my bandwidth use being fairly high and most of it being FTP rather than HTTP.
From: steve22 Nov 2012 15:27
To: ANT_THOMAS 7 of 36
Could you not host just the image on a local server? It can be captured and saved every minute (or even second) but they'd be no network activity unless someone was watching.
From: ANT_THOMAS22 Nov 2012 15:30
To: steve 8 of 36
I want to avoid that because my local server upload speed is slooooooow. If on the off chance I get the idea working properly there might be a couple of watchers, at least for a short time.

It is currently here and not exciting at all, unless you like watching out for the odd dumper truck - http://antthomas.co.uk/webcam/temp.php
From: steve22 Nov 2012 15:33
To: ANT_THOMAS 9 of 36
How about writing a script that'll run on your server and use PHP's file_get_contents to download it from your local server to your remote server? It could work, unless you want to do terrifying cron jobs, so on every refresh it downloads the image for the next refresh depending on the age of the file. If you get me.

That might get around the bandwidth ^.^
From: ANT_THOMAS22 Nov 2012 15:36
To: steve 10 of 36
Certainly a possibility but I think requesting an upload might be a bit slow.

The local system currently uploads the range of images every 20 seconds to the spare "100GB" free server I've found. Then there's a bit of Javascript to update the image and the date stamp every 20s.

From: Ken (SHIELDSIT)22 Nov 2012 16:29
To: ANT_THOMAS 11 of 36
Maybe get a box.com account and upload to there?  They are giving 50GB's away.  You'd have to see if they limit bandwidth though.

Oh, it's free, figured I'd mention that.
EDITED: 22 Nov 2012 16:31 by SHIELDSIT
From: ANT_THOMAS22 Nov 2012 16:35
To: Ken (SHIELDSIT) 12 of 36
I think I might have one already.

I was considering that with Dropbox, just need to look into bandwidth restrictions on public files.
From: Ken (SHIELDSIT)22 Nov 2012 16:37
To: ANT_THOMAS 13 of 36
Yeah, I only have 12GB on DP, but 50 on Box, and 25 on skydrive.  I'd look in to one of those I think.  If you can't find anything to your liking we could test it on my server.
From: Matt22 Nov 2012 17:36
To: ANT_THOMAS 14 of 36
Do you have SSH access to the server?

If you do, you might want to look into using rsync to only upload the image when it has changed.

If you were to use a image format that is predictably consistent in how it saves (I'm thinking one that doesn't use compression / isn't lossless, i.e. saving the same image to 2 different files creates 2 identical files - bitmap for instance) you could then get rsync to only transfer the parts of the image that have changed.

Only upload the largest resolution image, with compression on of course, and on the server use some shell scripting on a cron to copy and convert the bitmap to a web-friendly format and also create the smaller resolution images.

If you don't have SSH, forget I said anything.
From: ANT_THOMAS22 Nov 2012 17:39
To: Matt 15 of 36
I don't (giggle)

The image does change every 20 seconds though.

The script on my webcam machine is basically...

Take photo
Resize photo
Upload photo

This runs every 20 seconds.
From: Ken (SHIELDSIT)22 Nov 2012 17:46
To: ANT_THOMAS 16 of 36
What kind of router do you have?  With the new Asus routers (maybe new isn't the right word), with some Asus routers, like mine, you can connect an external drive and host it.  Asus also offers some cloud storage. Without looking I can't remember what you said the reason was for wanting to upload them and not put them on your server, but I can't imagine it's bandwidth because you're still uploading them.  So even if you're not using an Asus router, and if your router has usb ports you can install Tomato USB and host from there.  Just an idea...
From: ANT_THOMAS22 Nov 2012 17:53
To: Ken (SHIELDSIT) 17 of 36
It is bandwidth at my end. My upload pipe isn't fat enough for proper hosting.

Uploading a single (or a few) images every 20 seconds is fine. But having a number of people requesting said images every 20 seconds isn't.

Anyway, I'm using that free service I linked to up there for the time being, hopefully it's not too shit.
From: Ken (SHIELDSIT)22 Nov 2012 17:55
To: ANT_THOMAS 18 of 36
I must have missed that post, I'll check it out.  What is your upload speed?
From: ANT_THOMAS22 Nov 2012 17:55
To: Ken (SHIELDSIT) 19 of 36
~600 kbps
From: Ken (SHIELDSIT)22 Nov 2012 17:57
To: ANT_THOMAS 20 of 36
Ah yeah, that's not going to cut it is it?

So the AntThomas.co.uk is where you're putting them now?  Seems to work just fine!

And hey, it's raining and dark, it certainly shouldn't be dark this time of day!