python - StreamConsumedError() exception raised by "for chunk in response.iter_content(1024)" -


i'm not sure what's best way ask question,

but i'm writing script download content internet using request.get(), working fine, reason getting streamconsumederror() being raised, , i'm not sure why.

traceback (most recent call last): file "./pythondl-ver_0.0.4.py", line 90, in <module> chunk in response.iter_content(1024): file "/library/frameworks/python.framework/versions/2.7/lib/python2.7/site-packages/requests/models.py", line 766, in iter_content raise streamconsumederror() requests.exceptions.streamconsumederror

if need anymore information can add in. thanks.

while true:     count = count + 1     print 'testing ' + str(count) + '\n'     print url     url_2 = url.format(count = count)     print '////' + str(url_2) + ' loop ' + str(count) + "////\n"       print str(url_2) + ' testing url \n'      filename = posixpath.basename(url_2)     print str(filename) + ' testing filename \n'     response = requests.get(url_2, stream = true)     responsestring = str(response)      print str(responsestring) + 'testing res2 \n'      if responsestring == '<response [404]>':         print '......no more requests......\n'         break     elif responsestring == '<response [200]>':         print '......successful request....\n'     else:         break      print responsestring      while responsestring == '<response [200]>':         print 'testing while loop: ' + str(responsestring)         open(filename, 'wb') fp:             chunk in response.iter_content(1024):                 fp.write(chunk)      count += 1     print str(count) + ' = counting value'

here's loop things stopped working.

    while responsestring == '<response [200]>':         print 'testing while loop: ' + str(responsestring)         open(filename, 'wb') fp:             chunk in response.iter_content(1024):                 fp.write(chunk)

this loop never terminates, once responsestring <response [200]> stays forever, because nothing changes that.

the thing prevents loop go forever can read streaming response several times. when loop tries again response throws error.

the simplest fix replace while if

    if responsestring == '<response [200]>':         open(filename, 'wb') fp:             chunk in response.iter_content(1024):                 fp.write(chunk)

Comments

Post a Comment

Popular posts from this blog

javascript - Create a stacked percentage column -

i have collection of data : var data = [ {"p301a":"10","p301b":"7","p301c":"7","p301d":"3","p301e":"8","p301f":"8","p301g":"8","p301h":"8","p301i":"8","p301j":"8","p301k":"8","p301l":"8","p301m":"8","p301n":"8","p301o":"8","age":"31-40 years","profesion":"2","position":"2"}, {"p301a":"5","p301b":"4","p301c":"4","p301d":"4","p301e":"4","p301f":"4","p301g":"4","p301h":"4","p301i":"4","p301j":"4","p301k":"4...
Read more

Optimising Firebase database by automatically overwriting data -

in firebase app have following structure: posts latest all users can write posts > latest . want limit entries 20 posts save space in database. , posts beyond 20 unnecessary never shown on homepage. how can limit posts 20, when user writes posts > latest posts @ end drop off (be deleted) automatically? this sample should you. you need : const max_log_count = 20; exports.removeold = functions.database.ref('/posts/latest/{postid}').oncreate(event => { const parentref = event.data.ref.parent; return parentref.once('value').then(snapshot => { if (snapshot.numchildren() >= max_log_count) { let childcount = 0; const updates = {}; snapshot.foreach(function(child) { if (++childcount <= snapshot.numchildren() - max_log_count) { updates[child.key] = null; } }); // update parent. removes children. ...
Read more

javascript - Angular UI-Grid customTemplate directive causing rows to load slowly/? -

Image
each grid cell can have more 1 customtemplate loaded. have directive holds watcher on model_col_field listen cell updates on scroll when dom rows updated , recompile celltemplate . without watcher , $compile rows repeat on scroll, showing same set of rows. is there more angular way of approaching or way improve performance? relevant controller code: create gridoptions.data & gridoptions.columndefs : var customtemp = '<div ng-repeat="opt in grid.appscope.colvalueoptionsmodel" ng-init="tempstring=model_col_field.clazz+opt.key;"><comparison-directive dbo="model_col_field" attobj="opt" fff="grid.appscope.getattributetemplate(tempstring)" /></div>'; $scope.gridoptions.columndefs = [ { name:'yindex',pinnedleft:true,celltemplate: customtemp, displayname:'index'}, { name:'col1',celltemplate: customtemp}, { name:'col2',celltemplate: c...
Read more