Create a dynamic script to scrape multiple websites.

I anuluar Postuar 6 vite mё parё Paguhet në dorëzim
I anuluar Paguhet në dorëzim

Create a dynamic script in NodeJS to scrape 15+ sites using configurable json.

Requirement:

-Should be able to scrape lazy loading sites

-Download images

-Go through all pagination pages and scrape the child pages

-If element is a product link "click" that link and scrape that child page

-Get all below elements and retrieve correctly

See below for understanding how to set it up.

//Example configurable object

[

site1: {

url: "[login to view URL]",

parent: {

productList: "div.product-list",

product-link: "a.product-url",

product-name: ".product-info .product-title",

product-image: ".product-images img[src attribute]",

product-price: {

list-price: ".product-price span.old-price",

sale-price: ".product-price [login to view URL]"

}

pagination: "button.load-more-btn",

product-page: {

name: "form#product h1",

price: {

list-price: ".price .old",

sale-price: ".price new"

},

colors: "#options-articles li",

sizes: "#options-variants li",

description: ".description p",

images: {

main: "#images .product-image-box img[src attribute]",

thumbs: "#images .thumbs .product-thumbs li"

}

}

...

}

},

site2: {

Similar to above Site1

...

}

...

]

==================================================================================

//Product Class

Class Product() {

Product(name, link, image, price, colors, sizes, description) {

[login to view URL] = name;

[login to view URL] = image;

[login to view URL] = price;

[login to view URL] = colors;

[login to view URL] = sizes;

[login to view URL] = description;

}

}

===================================================================================

//Main Scraper Class

Class Scraper() {

Scraper(site) {

[login to view URL] = site;

}

getName(attr) {

return name value;

}

getLink(attr) {

return link value;

}

getMainImage(attr) {

return main value;

}

getThumbImages(attr) {

return thumb array;

}

getPrice(attr) {

return Price Object(list, sale);

}

getColor(attr) {

return Colors array;

}

getSizes(attr) {

return Sizes array;

}

getName(attr) {

return desc value;

}

hasPagination() {

if([login to view URL] element exist)

//scrape all pagination pages to the last page

}

downloadImage() {

if([login to view URL] element exist)

//download image

src = getImage();

download(src);

}

getProductInfo(product) {

//Go to product page and scrape using "product-page" attribute from json sample.

return {

name: String,

price {

old: String

new: String

},

images : {

main: String src,

thumbs: [array]

},

sizes: [array],

colors: [array],

description: String

}

}

getListOfProducts() {

//Use [login to view URL] to get listOfProducts

ArrayOfProducts = [];

for( items in listOfProducts ){

//create new product

product = new Product( getProductInfo() );

//add it to list

[login to view URL]( product );

}

}

}

Java JavaScript Node.js Python Nxjerrje të dhënash nga interneti

ID Projekti: #15659228

Rreth projektit

17 propozimet Projekti në distancë Aktiv 6 vite mё parё

17 profesionistë freelancer dërguan një ofertë mesatare prej $430 për këtë punë

mantislin

Dear sir, This is Lin and I am scraping expert, I have did many scraping projects, please check my reviews then you wil know. I have checked all details for your project, but there are some details that i wan Më shumë

$448 USD për 6 ditë
(348 Përshtypje)
7.7
schoudhary1553

Greeting, I have understood your Create a dynamic script to scrape multiple websites. task and can do it with your 100% satisfaction. Please ping me for more discussion. Relevant Skills and Experience I have more th Më shumë

$300 USD për 5 ditë
(83 Përshtypje)
6.8
Nada100200

Over 8 +years experience writing almost exclusively web scraping code. I've done it all. I can scrape all LinkedIn profile My languages in order of experience and use is Python, JavaScript, PHP. Python libraries ( sele Më shumë

$250 USD për 5 ditë
(15 Përshtypje)
5.5
aksharkthakkar

hello,we are team of experts and can do this project ,so lets discuss about it in detail and then start the project Relevant Skills and Experience Java, Javascript, node.js, Python, Web Scraping Proposed Milestones $ Më shumë

$555 USD për 8 ditë
(22 Përshtypje)
5.7
Rekhathakur

I have gone up through the job description I completely understood your all requirement, So I can easily Create a dynamic script to scrape multiple websites and able to start right now. Relevant Skills and Experience Më shumë

$555 USD për 10 ditë
(14 Përshtypje)
4.7
VirtualBrainInc

I have briefly read the description on scrap, and I can deliver as per the requirements however I need us to discuss for more clarity on the details, deadline and budget as well. Relevant Skills and Experience python Më shumë

$250 USD për 4 ditë
(9 Përshtypje)
4.2
anuragiitk

I am an IITK graduate and I have 11 years of experience in software development. I have 100% completion rate and I have finished all the projects with the highest level of customer satisfaction. Relevant Skills and Ex Më shumë

$555 USD për 10 ditë
(24 Përshtypje)
5.6
LuckyQ

Hello , My Name is Sam , I am Full Stack web and app developer . I'd read the Project details extremely carefully and I am absolutely sure that I can do the It very well . Relevant Skills and Experience To Know idea a Më shumë

$555 USD për 10 ditë
(18 Përshtypje)
4.2
divumanocha

Hi, I have gone through the job description I completely understood your all requirement and I am available with complete expertise in Python, node.js and past experience to meet your job requirement Relevant Skills Më shumë

$555 USD për 10 ditë
(5 Përshtypje)
3.9
Sheaflickz

I'm guessing you want to scrape these data to build a product or price comparison aggregator. I just finished a similar project although for Airline booking service using Nodejs. Relevant Skills and Experience Have b Më shumë

$366 USD për 10 ditë
(1 përshtypje)
3.1
adilsyed123

Hey, I'm interested in your project. Please send me a message so that we can discuss more. I can do trial work before hire me sir , will wait for your reply, Thanks, Syed

$250 USD për 5 ditë
(7 Përshtypje)
3.3
joshapstein

Been developing with MERN stack for 3+ years and would be able to finish this within 2 days easily. Let me know when to start Relevant Skills and Experience Been developing with MERN stack for 3+ years and would be ab Më shumë

$388 USD për 2 ditë
(1 përshtypje)
1.1