Extracting Teams on betting sites only gives me a limit of 13 teams
The betIn site i'm trying to extract data from is betIn.I have navigated to the section that includes a list of all teams that will currently be playing on the current day and i'm trying to extract the information about the teams but i'm currently only able to obtain 13 teams instead of all the teams.At first i thought storing the values in an Array was the problem hence i opted to move to an ArrayList but still bore the same problem Below is my code:
public class test {
public static void main(String args) {
System.setProperty("webdriver.chrome.driver", "/Users/user/Desktop/chromedriver");
WebDriver driver = new ChromeDriver();driver.navigate().to("https://sports.betin.co.ke/mobile#/dailyBundle/soccer/1-1000");
List<WebElement> rows = driver.findElements(By.cssSelector(".match-content.table-a.soccer"));
java.util.Iterator<WebElement> row_list = rows.iterator();
ArrayList<String> teams = new ArrayList<>();
ArrayList<String> bets = new ArrayList<>();
while(row_list.hasNext()){
WebElement rowItem = row_list.next();
String unnecessary = rowItem.findElement(By.cssSelector(".match-content__row--info")).getText();
String Content = rowItem.findElement(By.cssSelector(".match-content__info")).getText();
String relevantContent = Content.replace(unnecessary,"");
String bet = rowItem.findElement(By.cssSelector(".bets")).getText();
teams.add(relevantContent);
bets.add(bet);
}
System.out.println("These are the teams n"+ teams);
System.out.println("The size of teams is "+ teams.size());
System.out.println("These are the odds n"+ bets);
driver.close();
}
}
java selenium-webdriver web-scraping
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
add a comment |
The betIn site i'm trying to extract data from is betIn.I have navigated to the section that includes a list of all teams that will currently be playing on the current day and i'm trying to extract the information about the teams but i'm currently only able to obtain 13 teams instead of all the teams.At first i thought storing the values in an Array was the problem hence i opted to move to an ArrayList but still bore the same problem Below is my code:
public class test {
public static void main(String args) {
System.setProperty("webdriver.chrome.driver", "/Users/user/Desktop/chromedriver");
WebDriver driver = new ChromeDriver();driver.navigate().to("https://sports.betin.co.ke/mobile#/dailyBundle/soccer/1-1000");
List<WebElement> rows = driver.findElements(By.cssSelector(".match-content.table-a.soccer"));
java.util.Iterator<WebElement> row_list = rows.iterator();
ArrayList<String> teams = new ArrayList<>();
ArrayList<String> bets = new ArrayList<>();
while(row_list.hasNext()){
WebElement rowItem = row_list.next();
String unnecessary = rowItem.findElement(By.cssSelector(".match-content__row--info")).getText();
String Content = rowItem.findElement(By.cssSelector(".match-content__info")).getText();
String relevantContent = Content.replace(unnecessary,"");
String bet = rowItem.findElement(By.cssSelector(".bets")).getText();
teams.add(relevantContent);
bets.add(bet);
}
System.out.println("These are the teams n"+ teams);
System.out.println("The size of teams is "+ teams.size());
System.out.println("These are the odds n"+ bets);
driver.close();
}
}
java selenium-webdriver web-scraping
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
add a comment |
The betIn site i'm trying to extract data from is betIn.I have navigated to the section that includes a list of all teams that will currently be playing on the current day and i'm trying to extract the information about the teams but i'm currently only able to obtain 13 teams instead of all the teams.At first i thought storing the values in an Array was the problem hence i opted to move to an ArrayList but still bore the same problem Below is my code:
public class test {
public static void main(String args) {
System.setProperty("webdriver.chrome.driver", "/Users/user/Desktop/chromedriver");
WebDriver driver = new ChromeDriver();driver.navigate().to("https://sports.betin.co.ke/mobile#/dailyBundle/soccer/1-1000");
List<WebElement> rows = driver.findElements(By.cssSelector(".match-content.table-a.soccer"));
java.util.Iterator<WebElement> row_list = rows.iterator();
ArrayList<String> teams = new ArrayList<>();
ArrayList<String> bets = new ArrayList<>();
while(row_list.hasNext()){
WebElement rowItem = row_list.next();
String unnecessary = rowItem.findElement(By.cssSelector(".match-content__row--info")).getText();
String Content = rowItem.findElement(By.cssSelector(".match-content__info")).getText();
String relevantContent = Content.replace(unnecessary,"");
String bet = rowItem.findElement(By.cssSelector(".bets")).getText();
teams.add(relevantContent);
bets.add(bet);
}
System.out.println("These are the teams n"+ teams);
System.out.println("The size of teams is "+ teams.size());
System.out.println("These are the odds n"+ bets);
driver.close();
}
}
java selenium-webdriver web-scraping
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
The betIn site i'm trying to extract data from is betIn.I have navigated to the section that includes a list of all teams that will currently be playing on the current day and i'm trying to extract the information about the teams but i'm currently only able to obtain 13 teams instead of all the teams.At first i thought storing the values in an Array was the problem hence i opted to move to an ArrayList but still bore the same problem Below is my code:
public class test {
public static void main(String args) {
System.setProperty("webdriver.chrome.driver", "/Users/user/Desktop/chromedriver");
WebDriver driver = new ChromeDriver();driver.navigate().to("https://sports.betin.co.ke/mobile#/dailyBundle/soccer/1-1000");
List<WebElement> rows = driver.findElements(By.cssSelector(".match-content.table-a.soccer"));
java.util.Iterator<WebElement> row_list = rows.iterator();
ArrayList<String> teams = new ArrayList<>();
ArrayList<String> bets = new ArrayList<>();
while(row_list.hasNext()){
WebElement rowItem = row_list.next();
String unnecessary = rowItem.findElement(By.cssSelector(".match-content__row--info")).getText();
String Content = rowItem.findElement(By.cssSelector(".match-content__info")).getText();
String relevantContent = Content.replace(unnecessary,"");
String bet = rowItem.findElement(By.cssSelector(".bets")).getText();
teams.add(relevantContent);
bets.add(bet);
}
System.out.println("These are the teams n"+ teams);
System.out.println("The size of teams is "+ teams.size());
System.out.println("These are the odds n"+ bets);
driver.close();
}
}
java selenium-webdriver web-scraping
java selenium-webdriver web-scraping
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
edited Nov 13 '18 at 18:01
Bradley Juma
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
asked Nov 13 '18 at 4:09
Bradley JumaBradley Juma
7111
7111
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Did you notice that rows.size() is appearing zero always.
It is because page is not loaded completely but your code is getting executed.
If I run your code by applying some wait it works perfectly fine. I am able to get all teams.
Thread.sleep(10000) // wait for 10 seconds
List<WebElement> rows = testDriver.findElements(By.cssSelector(".match-content.table-a.soccer"));
Using sleep in between your code is not wise thing to do. So, to make sure your page is loaded completely check this out Selenium - How to wait until page is completely loaded
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53273671%2fextracting-teams-on-betting-sites-only-gives-me-a-limit-of-13-teams%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Did you notice that rows.size() is appearing zero always.
It is because page is not loaded completely but your code is getting executed.
If I run your code by applying some wait it works perfectly fine. I am able to get all teams.
Thread.sleep(10000) // wait for 10 seconds
List<WebElement> rows = testDriver.findElements(By.cssSelector(".match-content.table-a.soccer"));
Using sleep in between your code is not wise thing to do. So, to make sure your page is loaded completely check this out Selenium - How to wait until page is completely loaded
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
add a comment |
Did you notice that rows.size() is appearing zero always.
It is because page is not loaded completely but your code is getting executed.
If I run your code by applying some wait it works perfectly fine. I am able to get all teams.
Thread.sleep(10000) // wait for 10 seconds
List<WebElement> rows = testDriver.findElements(By.cssSelector(".match-content.table-a.soccer"));
Using sleep in between your code is not wise thing to do. So, to make sure your page is loaded completely check this out Selenium - How to wait until page is completely loaded
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
add a comment |
Did you notice that rows.size() is appearing zero always.
It is because page is not loaded completely but your code is getting executed.
If I run your code by applying some wait it works perfectly fine. I am able to get all teams.
Thread.sleep(10000) // wait for 10 seconds
List<WebElement> rows = testDriver.findElements(By.cssSelector(".match-content.table-a.soccer"));
Using sleep in between your code is not wise thing to do. So, to make sure your page is loaded completely check this out Selenium - How to wait until page is completely loaded
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
Did you notice that rows.size() is appearing zero always.
It is because page is not loaded completely but your code is getting executed.
If I run your code by applying some wait it works perfectly fine. I am able to get all teams.
Thread.sleep(10000) // wait for 10 seconds
List<WebElement> rows = testDriver.findElements(By.cssSelector(".match-content.table-a.soccer"));
Using sleep in between your code is not wise thing to do. So, to make sure your page is loaded completely check this out Selenium - How to wait until page is completely loaded
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
answered Nov 13 '18 at 11:26
paulpaul
1,618103874
1,618103874
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
add a comment |
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Thank you so much it works -Paul
– Bradley Juma
Nov 13 '18 at 13:20
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
Glad that it worked for you, accept answer.
– paul
Nov 13 '18 at 13:38
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53273671%2fextracting-teams-on-betting-sites-only-gives-me-a-limit-of-13-teams%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown