In the first one, I will talk about the analysis that I made to understand its structure and how could I prepare myself to write the scraping code. This post will start a series of 3 posts, where each one will talk about a part of the project. They are lxml (so you can parse code in HTML, XML and others), together with Xpath (Xpath is not a Python library, but a syntax to define parts in a XML document, and we will talk more about it later), and requests, used to make (guess what) requests in HTML pages and return the source code (among other things). To do our scraping, we will use two Python libraries that are installed with Anaconda. This is one good lesson: in Data Analysis, sometimes, the data is not available in an easy and accessible way, so it is necessary to develop another means to get it.

With modern languages and frameworks, it is always plausible to believe that there is a structure to the pages, patterns and repeatable structures that can be reproduced in your code. Basically, you access a web address (programatically) and, from the HTML source code, extract the information you want, through the identification of patterns and where in the code the information is. What is web scraping? Web Scraping is the act of extracting information from structured HTML documents (websites). If you want to check it out: Since Comixology does not have an API and since there is no Dataset to download or anything like that, I had to do web scraping. Comixology is a website that sells digital comics, with a huge catalog and a lot of publishers, big and small, and even independent ones. Since I had already used Comixology for comic reading, I opted to make an analysis of it. I like the subject for quite some time and I wanted to have a general idea of the comics and publishers (specially the giants, Marvel and DC Comics). In this post, i’ll talk about a project that I recently decided to do, involving an analysis of the digital comics market. Comixology - Digital Comics Analysis (Part 1 - analysis of site for web scraping)

Comixology website