Nrthugu

I have a paginated response from an URL, I want to keep on hitting the next page URL which I get from the previous response and keep on collecting items till I don't have a "nextPage" URL in my response. How to achieve this in a reactive way using spring boot WebClient from WebFlux with out blocking?

Request1: 



    GET /items

    response: 

    {

        items: [...]

        nextPage: "/items?page=2"

    }





    Request2: 



    GET /items?page=2

    response: 

    {

        items: [...]

        nextPage: "/items?page=3"

    }





    Request3: 



    GET /items?page=3

    response: 

    {

        items: [...]

        nextPage: null

    }

Here I have created mock urls
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items?page=2
https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items?page=3

How can I extract all Items from the above responses in a reactive way without blocking?

Using expand, this can be achieved.
Based on the mock urls provided by you.

public Mono<List<Item>> getItems() {

    String url = "https://karthikdivi.com/apps/paginatedReviews/withNextPageTokens/items";



    return fetchItems(url).expand(response -> {

        if (response.getNextPage() == null) {

            return Mono.empty();

        }

        return fetchItems(response.getNextPage());

    }).flatMap(response -> Flux.fromIterable(response.getItems())).collectList();

}



private Mono<Response> fetchItems(String url) {



         return client.get().uri(url).retrieve()

                    .bodyToMono(Response.class);

    }

You can achive a desired effect using exapnd:

@Test

public void usingExpand(){



    Request innerData = new Request(null);

    Request middleData = new Request(innerData);

    Request rootData = new Request(middleData);



    Mono.just(rootData)

            .expand( t -> Mono.justOrEmpty(t.netxPage))

            .flatMap( t -> Flux.fromIterable(t.items))

            .subscribe(System.out::println);



}



public static class Request {

    List<String> items = new ArrayList<>();

    Request netxPage;



    public Request(Request netxPage) {

        this.items.add(UUID.randomUUID().toString());

        this.items.add(UUID.randomUUID().toString());

        this.netxPage = netxPage;

    }

}

The above code shuld produce the following result:

dc78317c-5552-4723-90db-5392c67655be

32ff12bb-5be1-415e-b481-dab85d9157dd

cf1e3f36-a8e2-414d-90a2-7708eeedc5be

91a6bc14-a396-483d-a66a-80bb98dc1968

c95adae3-8e6f-489b-8a9d-4cea3080e150

d6f8fe01-2c50-4574-958c-ec675331bb25

Two UUID from each data object.