Python Beautiful Soup print specific lines within multiline containing string
How can I get/ print only the lines of a big multiline text within one <p>
tag containing a certain string? On the website the lines are realized with <br>
tags. There is no closing </p>
tag.
Basic structure of the website:
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
...
Let's assume I want to get/ print only the lines containing the words "coffee and cookies". So, in this case only the first and third "line"/ sentence of this <p>
should be printed.
I have Beautiful Soup 4.6.3 installed under Python 3.7.1.
findAll seems to be tag-orientated and return the whole <p>
, right? So how can I realize it? Maybe with regex or other pattern?
python html web-scraping beautifulsoup screen-scraping
add a comment |
How can I get/ print only the lines of a big multiline text within one <p>
tag containing a certain string? On the website the lines are realized with <br>
tags. There is no closing </p>
tag.
Basic structure of the website:
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
...
Let's assume I want to get/ print only the lines containing the words "coffee and cookies". So, in this case only the first and third "line"/ sentence of this <p>
should be printed.
I have Beautiful Soup 4.6.3 installed under Python 3.7.1.
findAll seems to be tag-orientated and return the whole <p>
, right? So how can I realize it? Maybe with regex or other pattern?
python html web-scraping beautifulsoup screen-scraping
add a comment |
How can I get/ print only the lines of a big multiline text within one <p>
tag containing a certain string? On the website the lines are realized with <br>
tags. There is no closing </p>
tag.
Basic structure of the website:
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
...
Let's assume I want to get/ print only the lines containing the words "coffee and cookies". So, in this case only the first and third "line"/ sentence of this <p>
should be printed.
I have Beautiful Soup 4.6.3 installed under Python 3.7.1.
findAll seems to be tag-orientated and return the whole <p>
, right? So how can I realize it? Maybe with regex or other pattern?
python html web-scraping beautifulsoup screen-scraping
How can I get/ print only the lines of a big multiline text within one <p>
tag containing a certain string? On the website the lines are realized with <br>
tags. There is no closing </p>
tag.
Basic structure of the website:
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
...
Let's assume I want to get/ print only the lines containing the words "coffee and cookies". So, in this case only the first and third "line"/ sentence of this <p>
should be printed.
I have Beautiful Soup 4.6.3 installed under Python 3.7.1.
findAll seems to be tag-orientated and return the whole <p>
, right? So how can I realize it? Maybe with regex or other pattern?
python html web-scraping beautifulsoup screen-scraping
python html web-scraping beautifulsoup screen-scraping
asked Nov 13 '18 at 11:07
user3087516user3087516
184
184
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
If I could understand your requirement correctly then the following snippet should get you there:
from bs4 import BeautifulSoup
htmlelem = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(htmlelem, 'html.parser')
for paragraph in soup.find_all('p'):
if not "coffee and cookies" in paragraph.text:continue
print(paragraph.get_text(strip=True))
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
add a comment |
Can you split on n ?
from bs4 import BeautifulSoup
html = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(html, 'html.parser')
for item in soup.select('p'):
r1 = item.text.split('n')
for nextItem in r1:
if "coffee and cookies" in nextItem:
print(nextItem)
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
add a comment |
convert bs4.element
to string using str()
then you can compare it with "coffee and cookies"
from bs4 import BeautifulSoup
html_doc = """<p style="line-height: 150%">
I need a big cup of coffee and cookies. <a href="aaa">aa</a>
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>"""
soup = BeautifulSoup(html_doc, 'html.parser')
paragraph = soup.find('p')
for p in paragraph:
if 'coffee and cookies' in str(p):
next_is_a = p.find_next_sibling('a')
if next_is_a:
print(p.strip() + ' ' + str(next_is_a))
else:
print(p.strip())
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it returnI need a big cup of coffee and cookies.
andI can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance
– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use.find_next_sibling()
see above
– ewwink
Nov 20 '18 at 13:01
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53279672%2fpython-beautiful-soup-print-specific-lines-within-multiline-p-containing-strin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
If I could understand your requirement correctly then the following snippet should get you there:
from bs4 import BeautifulSoup
htmlelem = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(htmlelem, 'html.parser')
for paragraph in soup.find_all('p'):
if not "coffee and cookies" in paragraph.text:continue
print(paragraph.get_text(strip=True))
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
add a comment |
If I could understand your requirement correctly then the following snippet should get you there:
from bs4 import BeautifulSoup
htmlelem = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(htmlelem, 'html.parser')
for paragraph in soup.find_all('p'):
if not "coffee and cookies" in paragraph.text:continue
print(paragraph.get_text(strip=True))
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
add a comment |
If I could understand your requirement correctly then the following snippet should get you there:
from bs4 import BeautifulSoup
htmlelem = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(htmlelem, 'html.parser')
for paragraph in soup.find_all('p'):
if not "coffee and cookies" in paragraph.text:continue
print(paragraph.get_text(strip=True))
If I could understand your requirement correctly then the following snippet should get you there:
from bs4 import BeautifulSoup
htmlelem = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(htmlelem, 'html.parser')
for paragraph in soup.find_all('p'):
if not "coffee and cookies" in paragraph.text:continue
print(paragraph.get_text(strip=True))
answered Nov 14 '18 at 13:10
SIMSIM
10.3k3743
10.3k3743
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
add a comment |
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
add a comment |
Can you split on n ?
from bs4 import BeautifulSoup
html = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(html, 'html.parser')
for item in soup.select('p'):
r1 = item.text.split('n')
for nextItem in r1:
if "coffee and cookies" in nextItem:
print(nextItem)
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
add a comment |
Can you split on n ?
from bs4 import BeautifulSoup
html = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(html, 'html.parser')
for item in soup.select('p'):
r1 = item.text.split('n')
for nextItem in r1:
if "coffee and cookies" in nextItem:
print(nextItem)
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
add a comment |
Can you split on n ?
from bs4 import BeautifulSoup
html = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(html, 'html.parser')
for item in soup.select('p'):
r1 = item.text.split('n')
for nextItem in r1:
if "coffee and cookies" in nextItem:
print(nextItem)
Can you split on n ?
from bs4 import BeautifulSoup
html = """
<p style="line-height: 150%">
I need a big cup of coffee and cookies.
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>
"""
soup = BeautifulSoup(html, 'html.parser')
for item in soup.select('p'):
r1 = item.text.split('n')
for nextItem in r1:
if "coffee and cookies" in nextItem:
print(nextItem)
answered Nov 14 '18 at 14:45
QHarrQHarr
31.7k82041
31.7k82041
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
add a comment |
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Thanks, for this example it also works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:55
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
Like soup.select('p,a') ? That would gather p and a tag elements.
– QHarr
Nov 20 '18 at 12:15
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
this prints only the text from a, right? What if I need also the href value?
– user3087516
Nov 20 '18 at 12:20
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
It will grab those elements then you would retrieve the href value from the a tag though it would be better to use soup.select('p, a[href]') stackoverflow.com/questions/1080411/…
– QHarr
Nov 20 '18 at 12:22
add a comment |
convert bs4.element
to string using str()
then you can compare it with "coffee and cookies"
from bs4 import BeautifulSoup
html_doc = """<p style="line-height: 150%">
I need a big cup of coffee and cookies. <a href="aaa">aa</a>
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>"""
soup = BeautifulSoup(html_doc, 'html.parser')
paragraph = soup.find('p')
for p in paragraph:
if 'coffee and cookies' in str(p):
next_is_a = p.find_next_sibling('a')
if next_is_a:
print(p.strip() + ' ' + str(next_is_a))
else:
print(p.strip())
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it returnI need a big cup of coffee and cookies.
andI can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance
– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use.find_next_sibling()
see above
– ewwink
Nov 20 '18 at 13:01
add a comment |
convert bs4.element
to string using str()
then you can compare it with "coffee and cookies"
from bs4 import BeautifulSoup
html_doc = """<p style="line-height: 150%">
I need a big cup of coffee and cookies. <a href="aaa">aa</a>
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>"""
soup = BeautifulSoup(html_doc, 'html.parser')
paragraph = soup.find('p')
for p in paragraph:
if 'coffee and cookies' in str(p):
next_is_a = p.find_next_sibling('a')
if next_is_a:
print(p.strip() + ' ' + str(next_is_a))
else:
print(p.strip())
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it returnI need a big cup of coffee and cookies.
andI can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance
– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use.find_next_sibling()
see above
– ewwink
Nov 20 '18 at 13:01
add a comment |
convert bs4.element
to string using str()
then you can compare it with "coffee and cookies"
from bs4 import BeautifulSoup
html_doc = """<p style="line-height: 150%">
I need a big cup of coffee and cookies. <a href="aaa">aa</a>
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>"""
soup = BeautifulSoup(html_doc, 'html.parser')
paragraph = soup.find('p')
for p in paragraph:
if 'coffee and cookies' in str(p):
next_is_a = p.find_next_sibling('a')
if next_is_a:
print(p.strip() + ' ' + str(next_is_a))
else:
print(p.strip())
convert bs4.element
to string using str()
then you can compare it with "coffee and cookies"
from bs4 import BeautifulSoup
html_doc = """<p style="line-height: 150%">
I need a big cup of coffee and cookies. <a href="aaa">aa</a>
<br>
I do not like tea with milk.
<br>
I can't live without coffee and cookies.
<br>"""
soup = BeautifulSoup(html_doc, 'html.parser')
paragraph = soup.find('p')
for p in paragraph:
if 'coffee and cookies' in str(p):
next_is_a = p.find_next_sibling('a')
if next_is_a:
print(p.strip() + ' ' + str(next_is_a))
else:
print(p.strip())
edited Nov 20 '18 at 13:00
answered Nov 13 '18 at 11:33
ewwinkewwink
11.7k22239
11.7k22239
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it returnI need a big cup of coffee and cookies.
andI can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance
– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use.find_next_sibling()
see above
– ewwink
Nov 20 '18 at 13:01
add a comment |
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it returnI need a big cup of coffee and cookies.
andI can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance
– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use.find_next_sibling()
see above
– ewwink
Nov 20 '18 at 13:01
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
I tried to use your code, the output is not the whole line but only: ) coffee and cookies
– user3087516
Nov 13 '18 at 12:58
it return
I need a big cup of coffee and cookies.
and I can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance– ewwink
Nov 13 '18 at 13:28
it return
I need a big cup of coffee and cookies.
and I can't live without coffee and cookies.
try it repl.it/repls/YouthfulRashInsurance– ewwink
Nov 13 '18 at 13:28
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
Thanks, for this example it works. What if we also have an <a href="...">-tag in the same line and it should be also printed out?
– user3087516
Nov 20 '18 at 11:53
use
.find_next_sibling()
see above– ewwink
Nov 20 '18 at 13:01
use
.find_next_sibling()
see above– ewwink
Nov 20 '18 at 13:01
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53279672%2fpython-beautiful-soup-print-specific-lines-within-multiline-p-containing-strin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown