python - Calling function on valid values of masked arrays -


i have 2 numpy masked arrays:

>>> x masked_array(data = [1 2 -- 4],              mask = [false false  true false],        fill_value = 999999) >>> y masked_array(data = [4 -- 0 4],              mask = [false  true false false],        fill_value = 999999)

if try divide x y, division operation not performed when 1 of operands masked, don't divide-by-zero error.

>>> x/y masked_array(data = [0.25 -- -- 1.0],              mask = [false  true  true false],        fill_value = 1e+20)

this works if define own division function, div:

>>> def div(a,b):     return a/b  >>> div(x, y) masked_array(data = [0.25 -- -- 1.0],              mask = [false  true  true false],        fill_value = 1e+20)

however, if wrap function vectorize, function called on masked values , error:

>>> np.vectorize(div)(x, y) traceback (most recent call last):   file "<input>", line 1, in <module>   file "/usr/lib64/python3.4/site-packages/numpy/lib/function_base.py", line 1811, in __call__     return self._vectorize_call(func=func, args=vargs)   file "/usr/lib64/python3.4/site-packages/numpy/lib/function_base.py", line 1880, in _vectorize_call     outputs = ufunc(*inputs)   file "<input>", line 2, in div zerodivisionerror: division 0

is there way can call function array arguments, , have function executed when of arguments unmasked?

the problem

calling function directly worked because, when call div(x,y), div's arguments a , b become maskedarrays x , y, , resulting code a/b x.__div__(y) (or __truediv__).

now, since x maskedarray, has intelligence perform division on maskedarray, following rules.

however, when vectorize it, div function not going see maskedarrays, scalars, couple of ints in case. so, when tries a/b in third items, 'something' zero, , error.

maskedarray's implementation seems based on re-implementing of numpy maskedarrays. see, example, have both numpy.log , numpy.ma.log. compare running both of them on maskedarray contains negative values. both return proper maskedarray, plain numpy version outputs complains dividing zero:

in [116]: x = masked_array(data = [-1, 2, 0, 4],      ...:              mask = [false, false,  true, false],      ...:        fill_value = 999999)  in [117]: numpy.log(x) /usr/bin/ipython:1: runtimewarning: divide 0 encountered in log   #!/usr/bin/python3 /usr/bin/ipython:1: runtimewarning: invalid value encountered in log   #!/usr/bin/python3 out[117]:  masked_array(data = [-- 0.6931471805599453 -- 1.3862943611198906],              mask = [ true false  true false],        fill_value = 999999)  in [118]: numpy.ma.log(x) out[118]:  masked_array(data = [-- 0.6931471805599453 -- 1.3862943611198906],              mask = [ true false  true false],        fill_value = 999999)

if run numpy.log version on plain list, return nan , inf invalid values, not throw error zerodivisionerror you're getting.

in [138]: = [1,-1,0]  in [139]: numpy.log(a) /usr/bin/ipython:1: runtimewarning: divide 0 encountered in log   #!/usr/bin/python3 /usr/bin/ipython:1: runtimewarning: invalid value encountered in log   #!/usr/bin/python3 out[139]: array([  0.,  nan, -inf])

simpler solution

with that, see 2 alternatives: first, simpler case listed, replace bad values no-op: 1 in div's case (note data different yours, there 0 didn't mark masked):

x = masked_array(data = [1, 2, 0, 4],              mask = [false, false,  true, false],        fill_value = 999999) y = masked_array(data = [4, 0, 0, 4],              mask = [false,  true, true, false],        fill_value = 999999) in [153]: numpy.vectorize(div)(x,y.filled(1)) out[153]:  masked_array(data = [0.25 2.0 -- 1.0],              mask = [false false  true false],        fill_value = 999999)

the problem approach filled values listed non-masked on result, not want.

better solution

now, div example, , want more complex behavior there not 'no-op' argument. in case, can numpy did log, , avoid throwing exception, instead returning specific value. in case, numpy.ma.masked. div's implementation becomes this:

in [154]: def div(a,b):      ...:     try:      ...:         return a/b      ...:     except exception e:      ...:         warnings.warn (str(e))      ...:         return numpy.ma.masked      ...:           ...:           in [155]: numpy.vectorize(div)(x,y) /usr/bin/ipython:5: userwarning: division 0   start_ipython() /usr/lib/python3.6/site-packages/numpy/lib/function_base.py:2813:     userwarning: warning: converting masked element nan.   res = array(outputs, copy=false, subok=true, dtype=otypes[0]) out[155]:  masked_array(data = [0.25 -- -- 1.0],              mask = [false  true  true false],        fill_value = 999999)

more generic solution

but perhaps have function , not want change it, or third-party. in case, use higher-order function:

in [164]: >>> def div(a,b):      ...:     return a/b      ...:   in [165]: def masked_instead_of_error (f):      ...:     def wrapper (*args, **kwargs):      ...:         try:      ...:             return f(*args, **kwargs)      ...:         except:      ...:             return numpy.ma.masked      ...:     return wrapper      ...:          in [166]: numpy.vectorize(masked_instead_of_error(div))(x,y) /usr/lib/python3.6/site-packages/numpy/lib/function_base.py:2813:             userwarning: warning: converting masked element nan.   res = array(outputs, copy=false, subok=true, dtype=otypes[0]) out[166]:  masked_array(data = [0.25 -- -- 1.0],              mask = [false  true  true false],        fill_value = 999999)

on implementations above, using warnings might or might not idea. may want restrict types of exceptions you'll catching returning numpy.ma.masked.

note masked_instead_of_error ready used decorator functions, not need use every time.


Comments

Post a Comment

Popular posts from this blog

php - Vagrant up error - Uncaught Reflection Exception: Class DOMDocument does not exist -

i've set bento/ubuntu-16.04 vagrant box using php 5.6, , getting following error quite few times when try , run shell script: "uncaught reflection exception: class domdocument not exist". i've tried adding following top of script (not together), each no success: "sudo apt-get install php5.6-xml", "sudo apt-get install php5-xml", "sudo apt-get install php-dom" i've tried resetting server "sudo service apache2 restart". does have suggestions?
Read more

vue.js - Create hooks for automated testing -

Image
qa guys complained can't automate frontend testing. because our html looks same outside. lazy hack did this <template> <div :role="$options.name"> ... </div> </template> <script> export default { name: 'vmcomponentname' } </script> which takes name script , applies html. in browser get: <div role="vmcomponentname"> ... </div> line :role="$options.name" goes in every component. is there dry er solutions? please share. additional details i'll explain in details looking for. imagine have vmusercreate form creates user. test case can create user . to automatically test without role, have use following selector: .wrapper > .wrapper > .wrapper > .submit-button the test extremely brittle (will break lot). if use roles can use [role="vmusercreateform"] .submit-button selector. many folds less brittle. so basically, i'm ...
Read more

Add new key value to json node in java -

i have json file : { "id":1, "name":"abc", "addressdetails": { "city":"newyork" } } i wanted add 1 more key-value ("pincode" : "414141") node 'addressdetails'. i tried using : objectmapper mapper = new objectmapper(); jsonnode root = mapper.readtree(new file("d://test.json")); objectnode node = mapper.createobjectnode(); node.with("addressdetails").put("pincode", "414141"); but it's not getting added, there way can ? you can try you should modify file path objectmapper mapper = new objectmapper(); objectnode nodes = mapper.readvalue(new file("d:\\test.txt"), objectnode.class); nodes.with("addressdetails").put("pincode", "414141"); mapper.writer().writevalue(new file("d:\\test.txt"), nodes);
Read more