Python :: chaining generators for API call -
i hitting api responses consists of many pages, say, 100.
this takes long, trying use generators
can yield
values each page , process until condition
met down pipeline.
import requests import json import itertools
i defining generator
make requests:
def api_query(query): page=1 while true: # requests, result in body: title = result["result"]["title"] primaryartist_name = result["result"]["primary_artist"]["name"] yield title, primaryartist_name page+=1
then, want filter first page response using other generator
:
def api_filter(): api_response = api_query('query') # 10 first results page1 = [api_response.next() in range(10)] # filtering page1 results yield results
lastly, test condition
, try return
desired values:
def api_print(): results = api_filter() page1 = list(it.islice(results, 5)) # check if page1 results satisfy given condition if condition: return page1 # or try next page else: page2 = list(it.islice(results.next(), 5)) return page2
however, @ last bit of code, can print page1; if code moves else
,
page2 = list(it.islice(results.next(), 5)) stopiteration
what doing wrong?
ps: sorry if non working example; full code long, using simplified question in order test concept of generators in case...
Comments
Post a Comment