Nrthugu

Simplification :

I have a website with links.

After clicking on each link , it goes to a new page that I need to visit the links ( by clicking , not navigating).

Visualization :

I've managed to do 99% percent of the job:

(async () =>

{

    const browser = await puppeteer.launch({headless: false});

    const page = await browser.newPage();

    let url = "https://www.mutualart.com/Artists";

    console.log(`Fetching page data for : ${url}...`);

    await page.goto(url);

    await page.waitForSelector(".item.col-xs-3");

    let arrMainLinks: ElementHandle = await page.$$('.item.col-xs-3 > a');   //get the main links



    console.log(arrMainLinks.length); // 16





    for (let mainLink of arrMainLinks) //foreach main link let's click it

    {

        let hrefValue =await (await mainLink.getProperty('href')).jsonValue();

        console.log("Clicking on " + hrefValue);

        await Promise.all([

                              page.waitForNavigation(),

                              mainLink.click({delay: 100})

                          ]);



        // let's get the sub links

        let arrSubLinks: ElementHandle = await page.$$('.slide >a');



        //let's click on each sub click

        for (let sublink of arrSubLinks)

        {

            console.log('██AAA');



            await Promise.all([

                                  page.waitForNavigation(),

                                  sublink.click({delay: 100})

                              ]);

            console.log('██BBB');



            // await page.goBack() 

            break; // for now ...

        }

        break;



    }



    await browser.close();

})();

So where is the problem ?

It reaches the ██AAA

But it never reaches ██BBB

And I get an error :

 C:temppuppeterr1app>node server2.js

Fetching page data for : https://www.mutualart.com/Artists...

16

Clicking on https://www.mutualart.com/Artist/Mr--Brainwash/9B3FED6BB81E6B8E

██AAA

(node:17200) UnhandledPromiseRejectionWarning: TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded

    at Promise.then (C:temppuppeterr1node_modulespuppeteerlibFrameManager.js:1230:21)

    at <anonymous>

  -- ASYNC --

    at Frame.<anonymous> (C:temppuppeterr1node_modulespuppeteerlibhelper.js:144:27)

    at Page.waitForNavigation (C:temppuppeterr1node_modulespuppeteerlibPage.js:599:49)

    at Page.<anonymous> (C:temppuppeterr1node_modulespuppeteerlibhelper.js:145:23)

    at Object.<anonymous> (C:temppuppeterr1appserver2.js:127:30)

    at step (C:temppuppeterr1appserver2.js:32:23)

    at Object.next (C:temppuppeterr1appserver2.js:13:53)

    at fulfilled (C:temppuppeterr1appserver2.js:4:58)

    at <anonymous>

    at process._tickCallback (internal/process/next_tick.js:188:7)

(node:17200) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

(node:17200) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Question:

What am I missing here ?

Why can't it reach the ██BBB ?

Complete code

Update:

https://github.com/GoogleChrome/puppeteer/issues/3535

Original Answer:

Update , I've managed to solve it but not via the regular way that I wanted.

It seems that there is a problem with ElementHandle. Which is why I've moved to pure DOM objects.

I'm still interested with a more intuitive solution rather by dealing with ElementHandle :

Anyway here is my solution :

(async () =>

{

    const browser = await puppeteer.launch({headless: false});

    const page = await browser.newPage();

    let url = "https://www.mutualart.com/Artists";

    console.log(`Fetching page data for : ${url}...`);

    await page.goto(url);

    await page.waitForSelector(".item.col-xs-3");



    let arrMainLinks = await page.evaluate(() =>

                                           {



                                               return Array.from(document.querySelectorAll('.item.col-xs-3 > a'));

                                           });

    console.log(arrMainLinks.length);

    for (let i = 0; i < arrMainLinks.length; i++) //get the main links

    {





        await page.evaluate((a) =>

                            {





                                return ([...document.querySelectorAll('.item.col-xs-3 > a')][a] as HTMLElement ).click();

                            }, i);

        await page.waitForNavigation();

        let arrSubLinks2 = await page.evaluate(() =>

                                               {

                                                   return Array.from(document.querySelectorAll('.slide>a'));

                                               });

        console.log(arrSubLinks2.length);

        for (let j = 0; j < arrSubLinks2.length; j++)

        {

            console.log('███AAA');

            await page.evaluate((a) =>

                                {



                                    return ([...document.querySelectorAll('.slide>a')][a] as HTMLElement) .click();

                                }, j);



            await page.waitForNavigation();

            let ddd: ElementHandle = await page.$$('.artist-name');

            console.log(ddd.length);



            console.log('███BBB');

            await page.waitFor(2000);

            await page.goBack();

            console.log('███CCC');



        }

        await page.waitFor(2000);

        await page.goBack();

    }

    await browser.close();

})();