Home / Unlabelled / Lk Scraper - An Fully Configurable Linkedin Scrape (Scrape Anything Within Linkedin)

Lk Scraper - An Fully Configurable Linkedin Scrape (Scrape Anything Within Linkedin)

Scrapes Any Linkedin Data

Installation
$ pip install git+git://github.com/jqueguiner/lk_scraper

Setup

Using Docker compose
$ docker-compose up -d
 $ docker-compose run lk_scraper python3

Using Docker only for selenium server
First, you need to run a selenium server
$ docker run -d -p 4444:4444 --shm-size 2g selenium/standalone-firefox:3.141.59-20200326
After running this command, from the browser navigate to your IP address followed by the port number and /grid/console. So the command will be http://localhost:4444/grid/console.

Retrieving Cookie

Browser-Independent:
Navigate to Linkedin.com and log in
 Open up the browser developer tools (Ctrl-Shift-I or right click -> inspect element)



Chrome:
Select the Application tab
 Under the Storage header on the left-hand menu, click the Cookies dropdown and select www.linkedin.com
 Find the li_at cookie, and double click the value to select it before copying

Firefox:
Select Storage tab
 Click the Cookies dropdown and select www.linkedin.com
 Find and copy the li_at value

Setting up the cookie

Method 1 : Setting the cookie in the config file
You can add your linkedin li_at cookie in the config file that is located in your home (~/.lk_scraper/config.yml) see


Method 2 : Setting the cookie at the Scraper level
from lk_scraper import Scraper
 li_at = "My_super_linkedin_cookie"
 scraper = Scraper(li_at=li_at)

Method 3 : Using Variable Environment
(Not implemented Yet)
$ export LI_AT="My_super_linkedin_cookie"

A full working example
run the jupyter notebook linkedin-example.ipynb

Usage
from lk_scraper import Scraper
 scraper = Scraper()

Company Scraping
from lk_scraper import Scraper
 scraper = Scraper()
 company = scraper.get_object(object_name='company', object_id='apple')

Profil Scraping
from lk_scraper import Scraper
 scraper = Scraper()
 profil = scraper.get_object(object_name='profil', object_id='jlqueguiner')



Via: feedproxy.google.com
Lk Scraper - An Fully Configurable Linkedin Scrape (Scrape Anything Within Linkedin) Lk Scraper - An Fully Configurable Linkedin Scrape (Scrape Anything Within Linkedin) Reviewed by Anónimo on 8:36 Rating: 5

Tags