Showing posts with label count. Show all posts
Showing posts with label count. Show all posts

Saturday, August 24, 2013

Scalding - WordCount example in local mode











Scala IDE based on eclipse
scalding on scala 2.9


How to run scalding on eclipse

1. install eclipse indigo ( preferable j2ee edition, but add maven plugin -m2e plugin fromupgrade repistory - Help ->Install new software))
2. In Help->Install New software -> add a site http://download.scala-ide.org/sdk/e37/scala29/stable/site
and install scala ide plugin
http://scala-ide.org/download/current.html
We will work with scalding template created byAmit Nithan
http://hokiesuns.blogspot.co.il/2012/07/running-your-scalding-jobs-in-eclipse.html
it already contains needed scalding dependencies
Onec you followed an article and scalding was tested in local mode your next step is to run it on real hadoop cluster.
In order to run Maven's package or other commands from eclipse do:
  • right-click project
  • run as
  • run configurations..
  • double click maven build (to create a new configuration)
  • give a name for configuration e.g. package
  • click variables
  • select "selected_resource_loc" and click ok
  • write your goal e.g. "package" or "clean package"
  • run
The next time when you want to package another project, you can use this configuration again:
  • right-click project
  • run as
  • run configurations..
  • select your maven configuration
  • run
ENJOY:)