Build an epoll based web crawler in C

Thanks for posting this project. I'm interested in working on it. Here is my experience I've been a socket developer for about 6 years now and i have developed numerous projects with sockets in C / C++.The Epoll event based architecture is modern and quite fast when handling new connections, if used well, can go a long way in achieving performance. Most developers fail to realize that , the Epoll event system runs on an infinite loop which can block in some cases if the read or write operation is heavy and a web crawler is a heavy task , so it could force the event loop to block. To cut the long story short, i can show you a more modern approach which can be used to even solve the C10M problem using both the Epoll Event Architecture and some additional tricks to achieve the best performance. Thanks

I have experience in both c programming and epoll sys(I haven't work with all oses) in. Please don't hesitate to contact me if you are interested and provide me full details on what you want to do and hopefully I can help you out with this project. Have a good night. Best regards, Vassily A.