Hi there,
My specialty is web crawling and I'm looking for customers who need this service. I enjoy what I do and I'm looking forward to work for you.
I use Python/Scrapy to do the job and sometimes I use PhantomJS with Selenium. I have developed a system capable of running spiders daily, with error handling, email notification, etc. If you need to use this service we can negotiate a monthly cost, this system currently runs 24 hours on a server located in New York City. This system also has an API to view the latest data without downloading the complete data file, because the system also stores the data in a database MongoDB it if necessary.
I can share the code of the spider, no problem, make not sense to share the code that runs this daily, the system makes use of Celery with Scrapyd. It is a little routine that runs every so often looking in the database for those spiders that should run, then queued them using Scrapyd API. Although I have no problem sharing this and detailing their operation.
The price of my offer is for the spider, $50. If you are interested in using the system, this is $50 a month, the first month would pay only the spider ($50).
In case you do not want to use the system, I can make the spider as executable python code and you can use cron to run this daily.
My weakness is English, mainly oral, usually I communicate in writing with international customers, if you do not mind then we can move forward.
Thanks,
Emiliano (Nano)