1) Write a function(s) that does the following:
For a given URL, get the source code and output text (as sometimes these differ)
Find any emails within and store these in the DB. No duplicates.
Find any URLs that are in sub folders only and store them - no duplicates. Also store domains that are not within the given URL.
Example url [login to view URL]
Store the following:
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
Do not store:
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
2) Write a function that will use the list of domains from the above function as a source and search one at a time according to oldest un-searched first. I'll sort that part, so you write a function for a given URL again. This time it will only store folders found on that domain, no duplicates. This should find the contact page from the home page. Once an email is found, store the email, and mark all entries with that domain as searched.
I'll write the rest of the page, create the database tables, and allow the URL to be inserted within a form to initiate the first function. The second function will check every minute or so via CRON to see if there are any domains/URLs needing to be searched and run from there. I have tried creating this before and had some success, I just want to start a fresh.
If you know of a better way to do this, I am open to ideas.
Hi sir,
I am scraping expert, I have did more than 350+ scraping project, please check my feedback then you will know.
Can we discuss more details about this project? then I will provide example data/script for you.
Thanks,
Lin
Hello sir
I hope you are doing well.
After reading your offer, this looks like a perfect fit for my skill sets so May I discuss with you for further details about this project?
I'm a web developer with over 6 years experience and very strong in PHP framework programming like CakePHP, Codeigniter, Laravel, Also very strong with MVC and OOP as well.
Greetings sir, I have completed many projects like this. your 100% satisfaction is assured if you allow me to serve. first chat with me where we can talk about briefly