apartments.com web scraper

grabbing all of the data I want from apartments.com with just a url

Apartments.com… without any research, it is probably safe to say it is the #1 way people found their current apartments. It is also a great underwriting tool - especially for rent comps.

Two things were annoying while underwriting with apartments.com:

1. Going through the apartment photos to then make a gut feeling call on how this apartment compared to the subject property.

2. Getting the same data (not always easy to find) over and over again for each comp

Regarding pain point #1: It is true that technology has come a long way, but it is also just as true that there is a lot of stuff that software is still garbage for. One of those is advanced image recognition. While I’m confident tools like Amazon Rekognition will eventually be impressive, for now I can do a much better job with my own eyes.

Regarding pain point #2: Luckily, the data scraping task is tedious, annoying, and stupid enough for software to tackle. I went ahead and wrote a web scraper that allows us to stay lazy.