Web Scraping Using Python

import requests
from bs4 import BeautifulSoup
import pandas as pd
Text = []
res = requests.get('https://en.wikipedia.org/wiki/Web_scraping')
soup = BeautifulSoup(res.text, 'html. Parser')
soup.select('mw-headline')
for i in soup.select('.mw-headline'):
print(i.text)
Text.append(i.text)
df = pd.DataFrame({'Text':Text}) 
df.to_csv('Web_Scrap.csv', index=False, encoding='utf-8')

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Easy Data Science roadmap in 2021

Data Science/Python — My journey so far

Creating Maps to Capture Data

About Me — Desmond (D.) Young

About Me — Desmond (D.) Young

Day 11 of #30DQUERY challenge

Data Labeling Contest: Crowdsourcing a scalable solution to generate labels for satellite imagery

The Importance of Data Integrity in COVID-19 Clinical Trials with Manisha Desai

Example of the Prediction recipe

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
YASH PATEL

YASH PATEL

More from Medium

Crawling Python

BUILT-IN FUNCTIONS IN PYTHON:

PYTHON FOR LOOPS

DATATYPES IN PYTHON-