Super simple hacker news scraper with pyquery.


virtualenv

virtualenv venv
source venv/bin/activate
pip install pyquery

hn.py

#!/usr/local/bin/python
from pyquery import PyQuery as pq

data = pq(url='https://news.ycombinator.com/')
news = zip(*[iter(data('.title'))]*2)

for pos, title in news:
	print pq(pos).text(), pq(title).text()

ln -s hn.py ~/bin/hn

hn | pbcopy

1. SpaceX lands rocket at sea second time after satellite launch ( phys.org )
2. Redis 3.2.0 is out ( antirez.com )
3. Why Wind Turbines Have Three Blades (2016) ( cringely.com )
4. TheAIGames – Develop your coding skills by competing in various AI competitions ( theaigames.com )
5. Go upgrade Xcode. Fix your Git security hole ( rachelbythebay.com )
6. FBI Harassing Core Tor Developer, Demand Meeting, but Refusing to Explain Why ( techdirt.com )
7. Uber's China Rival Close to Raising $2B in New Funding ( bloomberg.com )
8. 'Boaty McBoatface' polar ship named after David Attenborough ( bbc.co.uk )
9. GNU name system ( gnunet.org )
10. Every top 5 song from 1958 to 2016 ( polygraph.cool )
11. The first commercial gene therapy to provide an outright cure ( technologyreview.com )
12. Tool Recreating the “Decrypting Text” Effect Seen in the Movie “Sneakers” (2016) ( github.com )
13. A multitouch musical synth built in Typescript/React ( femurdesign.com )
14. Typed Python: new Mypy release ( mypy-lang.blogspot.com )
15. SpaceX: “Landing confirmed” ( twitter.com )
16. Humans are the highest energy apes, making us smarter but also fatter ( sciencemag.org )
17. Millennials and the Working Class Are Leaving America's Expensive Cities ( citylab.com )
18. Results from Candy Japan box design A/B test ( candyjapan.com )
19. When code is suspiciously fast: adventures in dead code elimination ( jeffq.com )
20. Fixing Vulnerabilities in the Zcash Protocol ( z.cash )
21. Fullstack Academy (YC S12) Is Hiring Instructors for Computer Science Saturdays ( workable.com )
22. XKCD Plots in Matplotlib: Going the Whole Way ( jakevdp.github.io )
23. Introducing yakbak: Record and playback HTTP interactions in Node.js ( flickr.net )
24. The void left by the parallel port ( medium.com )
25. National (JavaScript) Security Agency ( nsa.gov )
26. Coinbase, Ripple Close to Landing New York Bitcoin Licenses ( reuters.com )
27. Why James Baldwin Still Matters ( vanityfair.com )
28. Peer Street ( peerstreet.com )
29. The Productivity of Working Hours (2014) [pdf] ( iza.org )
30. US Senate website says use HTTP instead of HTTPS ( senate.gov )

🍺