Pentaho's "Hadoop File Input" (Spoon) always displays error when trying to read a file from HDFS -


i new pentaho , spoon , trying process file local hadoop node "hadoop file input" item in spoon (pentaho). problem every uri have tried far seems incorrect. don't know how connect hdfs pentaho.

to make clear, correct uri is:

hdfs://localhost:9001/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv

i know it's correct 1 because tested via command-line , works:

hdfs dfs -ls hdfs://localhost:9001/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv

so, setting environment field "static", here of uris have tried in spoon:

  • hdfs://localhost:9001/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv
  • hdfs://localhost:8020/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv
  • hdfs://localhost:9001
  • hdfs://localhost:9001/user/data/prueba_concepto/
  • hdfs://localhost:9001/user/data/prueba_concepto
  • hdfs:///

i tried solution garci garcía gives here: pentaho hadoop file input setting port 8020 , use following uri:

  • hdfs://catalin:@localhost:8020/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv

and changed 9001 , tried same technique:

  • hdfs://catalin:@localhost:9001/user/data/prueba_concepto/listadoproductos_2017_02_13-15_59_con_id.csv

but still nothing worked me ... everytime press mostrar fichero(s)... button (show file(s)), error pops saying that file cannot found.

i added "hadoop file input" image here.

thank you.

okey, solved this.

i had add new hadoop cluster tab "view" -> right click on hadoop cluster -> new

there had input hdfs hadoop configuration:

  • storage: hdfs
  • hostname: localhost
  • port: 9001 (by default 8020)
  • username: catalin
  • password: (no password)

after that, if hit "test" button, of tests fail. solved second 1 copying configuration properties had in local hadoop configuration file ($local_hadoop_home/etc/hadoop/core-site.xml) spoon's hadoop configuration file:

data-integration/plugins/pentaho-big-data-plugin/hadoop-configurations/hdp25/core-site.xml

after that, had modify data-integration/plugins/pentaho-big-data-plugin/plugin.properties , set property "active.hadoop.configuration" hdp25:

active.hadoop.configuration=hdp25

restart spoon , you're go.


Comments

Post a Comment

Popular posts from this blog

javascript - Create a stacked percentage column -

i have collection of data : var data = [ {"p301a":"10","p301b":"7","p301c":"7","p301d":"3","p301e":"8","p301f":"8","p301g":"8","p301h":"8","p301i":"8","p301j":"8","p301k":"8","p301l":"8","p301m":"8","p301n":"8","p301o":"8","age":"31-40 years","profesion":"2","position":"2"}, {"p301a":"5","p301b":"4","p301c":"4","p301d":"4","p301e":"4","p301f":"4","p301g":"4","p301h":"4","p301i":"4","p301j":"4","p301k":"4...
Read more

Optimising Firebase database by automatically overwriting data -

in firebase app have following structure: posts latest all users can write posts > latest . want limit entries 20 posts save space in database. , posts beyond 20 unnecessary never shown on homepage. how can limit posts 20, when user writes posts > latest posts @ end drop off (be deleted) automatically? this sample should you. you need : const max_log_count = 20; exports.removeold = functions.database.ref('/posts/latest/{postid}').oncreate(event => { const parentref = event.data.ref.parent; return parentref.once('value').then(snapshot => { if (snapshot.numchildren() >= max_log_count) { let childcount = 0; const updates = {}; snapshot.foreach(function(child) { if (++childcount <= snapshot.numchildren() - max_log_count) { updates[child.key] = null; } }); // update parent. removes children. ...
Read more

javascript - Angular UI-Grid customTemplate directive causing rows to load slowly/? -

Image
each grid cell can have more 1 customtemplate loaded. have directive holds watcher on model_col_field listen cell updates on scroll when dom rows updated , recompile celltemplate . without watcher , $compile rows repeat on scroll, showing same set of rows. is there more angular way of approaching or way improve performance? relevant controller code: create gridoptions.data & gridoptions.columndefs : var customtemp = '<div ng-repeat="opt in grid.appscope.colvalueoptionsmodel" ng-init="tempstring=model_col_field.clazz+opt.key;"><comparison-directive dbo="model_col_field" attobj="opt" fff="grid.appscope.getattributetemplate(tempstring)" /></div>'; $scope.gridoptions.columndefs = [ { name:'yindex',pinnedleft:true,celltemplate: customtemp, displayname:'index'}, { name:'col1',celltemplate: customtemp}, { name:'col2',celltemplate: c...
Read more