Wednesday, January 21, 2015

Scraping multiple stock sites for tickers and recommendations

Installation:
Python 2.7 and lxml package (net)

Script:
from lxml import html
import requests
import sys

page = requests.get('http://www.stock1010.com/');
tree = html.fromstring(page.text)

tickers = tree.xpath('//table[@id="AutoNumber1"]//tr[2]/td/p/font/a')

for row in tickers:
    sys.stdout.write(row.text)
    sys.stdout.write ('  ')
     
    ticker = row.text
    swingtradepage = requests.get('http://www.swingtradebot.com/equities/' + ticker)
    swingtree = html.fromstring(swingtradepage.text)
    grade = swingtree.xpath('//*[@id="content"]//div/div[1]/div[2]/div[1]/table/tr/td[1]/img')
    value = swingtree.xpath('//*[@id="content"]//div/div[1]/div[2]/div[1]/table/tr/td[2]')
   
    for gr in grade:
         sys.stdout.write( gr.attrib['alt'] + '  ')
    for val in value:
         sys.stdout.write (val.text + '  ')

    unclestock = requests.get('http://www.unclestock.com/output?s=' + ticker)
    uncletree  = html.fromstring(unclestock.text)
    advice = uncletree.xpath('//table[1]//tr[1]/td[3]')

    ##print advice

    for ad in advice:
        print ad.text_content()

Help:
Chrome => Inspect Element
Chrome => Scraper

Take the list of tickers from stock1010.com, then pull the information of stock grade
from swingtradebot.com
and
recommendation from unclestock.com