Salmon Run

Swimming upstream on the technology tide, one technology at a time. A collection of articles, tips, and random musings on application development and system design.

Showing posts with label crawler. Show all posts
Showing posts with label crawler. Show all posts
Saturday, August 02, 2014

Quick and Dirty Web Crawling with ScraPy

›
I haven't crawled using Python before - we use Apache Nutch (NutchGORA actually) for large crawls, and its been a while since I had to...
4 comments:
Friday, February 10, 2012

Nutch/GORA - Using a sitemap to seed a site

›
The genesis of this "feature" was due to my misunderstanding of how Nutch works. So far I've been running fairly small batches...
2 comments:
›
Home
View web version

About me

My photo
Sujit Pal
I am a programmer interested in Semantic Search, Ontology, Natural Language Processing and Machine Learning. My programming languages of choice are Java, Scala, and Python. I love solving problems and exploring different possibilities with open source tools and frameworks.
View my complete profile
Powered by Blogger.