Scraping Xing with code and without code (one is 100x faster)

Xing scraping with Xingzap

Hallo 🇩🇪 ! 

Scraping is probably my favorite discipline in the Growth field and that’s why I’ll explain today how to do a little Xing scraping in different ways.

First I’ll explain how to scrape Xing with a somewhat advanced stack because Xing’s technology doesn’t allow for the usual scraping stack like Python’s Scrapy or BeautifulSoup.

Secondly, I’ll explain how you can do it yourself with XingZap and no other tools or skills, saving you at worst hours of development time and debugging.

If you don't want to know how to do the technical dive, simply scroll down or just get your XingZap account ! (you get 1000 credits free to get started.

Let’s get into it.

Scraping Xing with Python, Selenium and Chrome

Here’s what you need to get started smoothly:

Before we do anything we’ll need to login programmatically to your account but I’ll start with a disclaimer first:

Please be aware that there can be disruptions if you login too often from different IPs/locations. 
Xing (like Linkedin) has their own policy on account banning, and their word is final.
XingZap and I cannot be held responsible for the actions you decided to undertake with your own account

With that out of the way:

Here’s what the final logging function could look like raw:

This was only the beginning !

We still need to find the selectors for the data we want (name, position, company...) and output it.For the sake of simplicity, I’ll just show how I would get the first page of results of a given Xing search:

xing search results
unformatted result from selenium scraping
python scraping result with beautifulsoup
scraped xing urls

Putting it all together, it would look like this after amending the first script:

This script is functional as of December 2021, but I should note that nothing guarantees that it will still hold up in the future as you may already know.

With start, you’ll have to test and learn the rest on your own to see the limitations and tailor the Xing scraper script to your needs.

There are obviously more steps for this script to be ready for production : we still need to implement the paging, improve the input and the outputs, add human behavior and waiting times and finally make sure our errors are handled with screenshots in case Xing changes their layout.

You can take all this into account or simply read the next section to get started in seconds.

Xing scraping with XingZap

Here’s what you need to get started:

To get started, create an account on XingZap.

To replicate what I did when scraping, type fullstack developer and click search:

XingZap search for xing scraping

And boom you're already done 🎉 !

Xingzap result of Xing scraping

You can already download the export file which is already formatted.

Everything was done in a blink and that's why we developed XingZap .

If you want to try it out, we offer 1000 search results for free on signup 🚀

Until next time,

Tschüss !

Anas El Mhamdi
Anas El Mhamdi
Co-Founder and Growth @Xingzap