How to interact with location specific popups with selenium and python
I'm trying to build a web scraper with selenium that pulls pricing and nutrition data from https://www.hannaford.com/home.jsp for a school project. The difficulty is that pricing data is store specific. To solve this I have used selenium to enter the users zip-code on the website. Afterwards, a pop up window opens with five of the nearest stores. The program needs to select a store from these locations.
I've copied the x-path of the button that selects the first store and set it to be clicked, but selenium still does not continue past the popup.
Here is my code:
hannaford =
"https://www.hannaford.com/home.jsp"
#opens website
driver.get(hannaford)
driver.find_element_by_xpath("""//*[@id="node01"]/a""").click()
driver.find_element_by_xpath("""//*[@id="pageContentWrapperInner"]/div[2]/div/div[1]/p[2]/a""").click()
inputElement = driver.find_element_by_xpath("""//*[@id="cityStateZipLayer"]""")
inputElement.send_keys(zip)
inputElement.send_keys(Keys.ENTER)
#attempt to select store in popup
driver.find_element_by_xpath("""//*[@id="glo-store-results-layer-store-list"]/div[1]/span/form/a""").click()
and here is the html associated with the link I'm trying to click:
<span class="fr rightSide">
<form class="fr" action="/custserv/save_user_store.cmd" method="post" name="selectThisStoreForm"><input type="hidden" name="form_state" value="selectThisStoreForm"><input name="storeId" type="hidden" value="21185" autofillparam="ON"><a href="" onclick="javascript:saveUserStoreNoClose('21185');return false;" class="altLink">
<input class="shopNow" type="image" autofillparam="ON" src="/assets/hf/assets/images/buttons/btn_shopNow.gif" border="0" alt="Shop Now"></a>
</form></span>
How could I resolve this issue. Is it even possible to select dynamic elements like this in selenium?
python html css selenium web-scraping
asked Nov 12 '18 at 0:00
Black_lotus
32
add a comment |
I'm trying to build a web scraper with selenium that pulls pricing and nutrition data from https://www.hannaford.com/home.jsp for a school project. The difficulty is that pricing data is store specific. To solve this I have used selenium to enter the users zip-code on the website. Afterwards, a pop up window opens with five of the nearest stores. The program needs to select a store from these locations.
I've copied the x-path of the button that selects the first store and set it to be clicked, but selenium still does not continue past the popup.
Here is my code:
hannaford =
"https://www.hannaford.com/home.jsp"
#opens website
driver.get(hannaford)
driver.find_element_by_xpath("""//*[@id="node01"]/a""").click()
driver.find_element_by_xpath("""//*[@id="pageContentWrapperInner"]/div[2]/div/div[1]/p[2]/a""").click()
inputElement = driver.find_element_by_xpath("""//*[@id="cityStateZipLayer"]""")
inputElement.send_keys(zip)
inputElement.send_keys(Keys.ENTER)
#attempt to select store in popup
driver.find_element_by_xpath("""//*[@id="glo-store-results-layer-store-list"]/div[1]/span/form/a""").click()
and here is the html associated with the link I'm trying to click:
<span class="fr rightSide">
<form class="fr" action="/custserv/save_user_store.cmd" method="post" name="selectThisStoreForm"><input type="hidden" name="form_state" value="selectThisStoreForm"><input name="storeId" type="hidden" value="21185" autofillparam="ON"><a href="" onclick="javascript:saveUserStoreNoClose('21185');return false;" class="altLink">
<input class="shopNow" type="image" autofillparam="ON" src="/assets/hf/assets/images/buttons/btn_shopNow.gif" border="0" alt="Shop Now"></a>
</form></span>
How could I resolve this issue. Is it even possible to select dynamic elements like this in selenium?
python html css selenium web-scraping
asked Nov 12 '18 at 0:00
Black_lotus
32
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
What error do you get with your code? If it isNoSuchElementException, you should try using WebDriverWait to wait for the element to load.
– Kamal
Nov 12 '18 at 7:46
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40
add a comment |
I'm trying to build a web scraper with selenium that pulls pricing and nutrition data from https://www.hannaford.com/home.jsp for a school project. The difficulty is that pricing data is store specific. To solve this I have used selenium to enter the users zip-code on the website. Afterwards, a pop up window opens with five of the nearest stores. The program needs to select a store from these locations.
I've copied the x-path of the button that selects the first store and set it to be clicked, but selenium still does not continue past the popup.
Here is my code:
hannaford =
"https://www.hannaford.com/home.jsp"
#opens website
driver.get(hannaford)
driver.find_element_by_xpath("""//*[@id="node01"]/a""").click()
driver.find_element_by_xpath("""//*[@id="pageContentWrapperInner"]/div[2]/div/div[1]/p[2]/a""").click()
inputElement = driver.find_element_by_xpath("""//*[@id="cityStateZipLayer"]""")
inputElement.send_keys(zip)
inputElement.send_keys(Keys.ENTER)
#attempt to select store in popup
driver.find_element_by_xpath("""//*[@id="glo-store-results-layer-store-list"]/div[1]/span/form/a""").click()
and here is the html associated with the link I'm trying to click:
<span class="fr rightSide">
<form class="fr" action="/custserv/save_user_store.cmd" method="post" name="selectThisStoreForm"><input type="hidden" name="form_state" value="selectThisStoreForm"><input name="storeId" type="hidden" value="21185" autofillparam="ON"><a href="" onclick="javascript:saveUserStoreNoClose('21185');return false;" class="altLink">
<input class="shopNow" type="image" autofillparam="ON" src="/assets/hf/assets/images/buttons/btn_shopNow.gif" border="0" alt="Shop Now"></a>
</form></span>
How could I resolve this issue. Is it even possible to select dynamic elements like this in selenium?
python html css selenium web-scraping
asked Nov 12 '18 at 0:00
Black_lotus
32
I'm trying to build a web scraper with selenium that pulls pricing and nutrition data from https://www.hannaford.com/home.jsp for a school project. The difficulty is that pricing data is store specific. To solve this I have used selenium to enter the users zip-code on the website. Afterwards, a pop up window opens with five of the nearest stores. The program needs to select a store from these locations.
I've copied the x-path of the button that selects the first store and set it to be clicked, but selenium still does not continue past the popup.
Here is my code:
hannaford =
"https://www.hannaford.com/home.jsp"
#opens website
driver.get(hannaford)
driver.find_element_by_xpath("""//*[@id="node01"]/a""").click()
driver.find_element_by_xpath("""//*[@id="pageContentWrapperInner"]/div[2]/div/div[1]/p[2]/a""").click()
inputElement = driver.find_element_by_xpath("""//*[@id="cityStateZipLayer"]""")
inputElement.send_keys(zip)
inputElement.send_keys(Keys.ENTER)
#attempt to select store in popup
driver.find_element_by_xpath("""//*[@id="glo-store-results-layer-store-list"]/div[1]/span/form/a""").click()
and here is the html associated with the link I'm trying to click:
<span class="fr rightSide">
<form class="fr" action="/custserv/save_user_store.cmd" method="post" name="selectThisStoreForm"><input type="hidden" name="form_state" value="selectThisStoreForm"><input name="storeId" type="hidden" value="21185" autofillparam="ON"><a href="" onclick="javascript:saveUserStoreNoClose('21185');return false;" class="altLink">
<input class="shopNow" type="image" autofillparam="ON" src="/assets/hf/assets/images/buttons/btn_shopNow.gif" border="0" alt="Shop Now"></a>
</form></span>
How could I resolve this issue. Is it even possible to select dynamic elements like this in selenium?
python html css selenium web-scraping
python html css selenium web-scraping
asked Nov 12 '18 at 0:00
Black_lotus
32
asked Nov 12 '18 at 0:00
Black_lotus
32
asked Nov 12 '18 at 0:00
Black_lotus
32
asked Nov 12 '18 at 0:00
Black_lotus
32
asked Nov 12 '18 at 0:00
Black_lotus
32
32
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
What error do you get with your code? If it isNoSuchElementException, you should try using WebDriverWait to wait for the element to load.
– Kamal
Nov 12 '18 at 7:46
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40
add a comment |
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
What error do you get with your code? If it isNoSuchElementException, you should try using WebDriverWait to wait for the element to load.
– Kamal
Nov 12 '18 at 7:46
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
What error do you get with your code? If it is
NoSuchElementException, you should try using WebDriverWait to wait for the element to load.– Kamal
Nov 12 '18 at 7:46
What error do you get with your code? If it is
NoSuchElementException, you should try using WebDriverWait to wait for the element to load.– Kamal
Nov 12 '18 at 7:46
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53254463%2fhow-to-interact-with-location-specific-popups-with-selenium-and-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53254463%2fhow-to-interact-with-location-specific-popups-with-selenium-and-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The popup might not be loaded by the time you try to interact with it. Try adding a time.sleep(1) inbetween opening the popup and interacting with it.
– Brandon Lyons
Nov 12 '18 at 0:12
What error do you get with your code? If it is
NoSuchElementException, you should try using WebDriverWait to wait for the element to load.– Kamal
Nov 12 '18 at 7:46
@Black_lotus What are the Manual Steps which you are trying to Automate?
– DebanjanB
Nov 12 '18 at 8:13
Thank you, @Brandon Lyons, using time.sleep(1) solved my issue!
– Black_lotus
Nov 12 '18 at 17:40