Tools for scraping appic.org for my wife

Scrapes the first page of some appic search results and downloads details from each program page into out.csv which can be opened with excel.

Setup

install virtualenv + virtualenvwrapper

curl -s https://raw.github.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | bash

make and configure a virtualenv

mkvirtualenv appic --no-site-packages
workon appic
pip install -r path/to/appic/requirements.txt

Usage

cd path/to/appic

create out.csv in the same directory with details of programs which match search_url in scrape.py

python scrape.py 

same as above with results limited to first 5 programs

python scrape.py 5

Todo

• scrape more than the first page of results
• scrape the remainder of each program page
  • Training Opportunities and below