java - spring batch one step job in parallel -
i'm writing batch application based on spring batch. goal of application read csv file, , each entry, make http call endpoint.
i've followed spring.io tutorial , looks , works great.
@configuration @enablebatchprocessing public class migrationbatchconfiguration { @autowired public jobbuilderfactory jobbuilderfactory; @autowired public stepbuilderfactory stepbuilderfactory; @bean public flatfileitemreader<migrationitem> reader() { flatfileitemreader<migrationitem> reader = new flatfileitemreader<>(); reader.setresource(new classpathresource("extract-learningpaths.csv")); reader.setlinemapper(new defaultlinemapper<migrationitem>() {{ setlinetokenizer(new delimitedlinetokenizer() {{ setnames(new string[] { "id", "userid", "studylanguage", "addonids", "addoncodes", "score", "scorecode", "resultid" }); setdelimiter(";"); }}); setfieldsetmapper(new beanwrapperfieldsetmapper<migrationitem>() {{ settargettype(migrationitem.class); }}); }}); return reader; } @bean public migrationitemprocessor migrationitemprocessor() { return new migrationitemprocessor(); } @bean public job migrationjob(migrationcompletionnotificationlistener listener) { return jobbuilderfactory.get("migrationjob") .incrementer(new runidincrementer()) .listener(listener) .flow(readandprocess()) .end() .build(); } @bean public step readandprocess() { return stepbuilderfactory.get("readandprocess") .<migrationitem, migrationitem> chunk(10) .reader(reader()) .processor(migrationitemprocessor()) .build(); }
}
this version waits processor completed before starting process of next item. each item processed independently , wondering best approach achieve ?
i added asyncitemprocessor simpleasynctaskexecutor follow, best approach ?
@bean public step readandprocess() { return stepbuilderfactory.get("readandprocess") .<migrationitem, void>chunk(10) .reader(reader()) .processor(asyncitemprocessor()) .build(); } @bean public itemprocessor asyncitemprocessor() { asyncitemprocessor<migrationitem, void> asyncitemprocessor = new asyncitemprocessor<>(); asyncitemprocessor.setdelegate(migrationitemprocessor()); asyncitemprocessor.settaskexecutor(taskexecutor()); return asyncitemprocessor; } @bean public taskexecutor taskexecutor() { simpleasynctaskexecutor taskexecutor = new simpleasynctaskexecutor(); return taskexecutor; }
and if want add writer csv after processing ? asyncitemprocessor returns futuretask not want write, best approach ?
Comments
Post a Comment