Tools required to build and run Apache Atlas on Eclipse
- Download Maven 3 3 9
- Apache Maven 3.0.5 Download
- Apache Maven 3.0.4 Download
- Apache Maven 3.3.9 Download For Mac
- Apache Maven 3.3.3 Download
These instructions are provided as-is. They worked at a point in time; other variants of software may work. These instructions may become stale if the build dependencies change.
Now I have a folder called apache-maven-3.3.9, and if I open it up, I'll see there's a bin folder, a boot folder, a conf folder, and a lib folder. At this point, I've completed my download. The instructions for using Maven depends on whether you're running a Windows machine or a Mac OS or Linux machine.
They have been shown to work on 19th of December 2016.
To build, run tests, and debug Apache Atlas, the following software is required:
- Extract the distribution archive, i.e.apache-maven-3.3.9-bin.tar.gz to the directory you wish to install Maven 3.3.9. The subdirectory apache-maven-3.3.9 will be created from the archive. Optional: Add the MAVENOPTS environment variable to specify JVM properties, e.g. Export MAVENOPTS='-Xms256m -Xmx512m'. This environment variable can be used.
- Download Apache Maven. 2.1 Visit Maven official website, download the Maven zip file, for example: apache-maven-3.6.0-bin.zip.
Java
- Download and install a 1.8 Java SDK
- Set JAVA_HOME system environment variable to the installed JDK home directory
- Add JAVA_HOME/bin directory to system PATH
Atlas command line tools are written in Python.
- Download and install Python version 2.7.7
- For Mac, we used 2.7.11
- Add Python home directory to system PATH
- Download and install Maven 3.3.9
- Set the environment variable M2_HOME to point to the maven install directory
- Add M2_HOME/bin directory to system PATH e.g. C:UsersIBM_ADMINDocumentsSoftwareapache-maven-3.3.9bin
- Install Git
- Add git bin directory to the system PATH e.g. C:Program Files (x86)Gitbin
- Install Eclipse Neon (4.6)
- The non-EE Neon for iOS from eclipse.org has been proven to work here.
- Install the Scala IDE, TestNG, and m2eclipse-scala features/plugins as described below.
Some of the Atlas source code is written in the Scala programming language. The Scala IDE feature is required to compile Scala source code in Eclipse.
- In Eclipse, choose Help - Install New Software.
- Click Add.. to add an update site, and set Location to http://download.scala-ide.org/sdk/lithium/e44/scala211/stable/site
- Select Scala IDE for Eclipse from the list of available features
- Restart Eclipse after install
- Set the Scala compiler to target the 1.7 JVM: Window - Preferences - Scala - Compiler, change target to 1.7
Atlas tests use the TestNG framework, which is similar to JUnit. The TestNG plug-in is required to run TestNG tests from Eclipse.
- In Eclipse, choose Help - Install New Software.
- Click Add.. to add an update site, and set Location to http://beust.com/eclipse-old/eclipse_6.9.9.201510270734
- Choose TestNG and continue with install
- Restart Eclipse after installing the plugin
- In Window - Preferences - TestNG, uncheck 'Use project TestNG jar'
- In Eclipse, choose Help - Install New Software.
- Click Add.. to add an update site, and set Location to http://alchim31.free.fr/m2e-scala/update-site/
- Choose Maven Integration for Scala IDE, and continue with install
- Restart Eclipse after install
- In Window - Preferences -Maven - Errors/Warnings, set Plugin execution not covered by lifecycle configuration to Warning
a. File - Import - Maven - Existing Maven Projects b. Browse to your Atlas folder c. Uncheck the root project and non-Java projects such as dashboardv2, docs and distro, then click Finish
On the Mac, the Maven import fails with message
Install http://download.eclipse.org/tools/ajdt/46/dev/update and rerun. The Maven AspectJ should plugin install - allowing the references to Aspects in Maven to be resolved.
d. In the atlas-typesystem, atlas-repository, hdfs-model, and storm-bridge projects, add the src/main/scala and src/test/scala (if available) directories as source folders. Note: the hdfs-model and storm-bridge projects do not have the src/test/scala folder.
Right-click on the project, and choose Properties.
Click the Java Build Path in the left-hand panel, and choose the Source tab.
Click Add Folder, and select the src/main/scala and src/test/scala directories.
Only the atlas-repository and atlas-type system projects have Scala source folders to update.
e. Select atlas-typesystem, atlas-repository, hdfs-model, and storm-bridge projects, right-click, go to the Scala menu, and choose ‘Set the Scala Installation’.
f. Choose Fixed Scala Installation: 2.11.8 (bundled) , and click OK.
g. Restart Eclipse
h. Choose Project - Clean, select Clean all projects, and click OK.
Some projects may not pick up the Scala library – if this occurs, quick fix on those projects to add in the Scala library – projects atlas-typesystem, atlas-repository, hdfs-model, storm-bridge and altas-webapp.
You should now have a clean workspace.
Sample Bash scripts to help mac users
You will need to change some of these scripts to point to your installation targets.
- Run this script to setup your command line build environment
- If you do not want to set Java 8 as your system java, you can use this bash script to setup the environment and run Eclipse (which you can drop in Applications and rename to neon).
Prerequisites
- Java JDK 1.7
- Apache Maven 3.3.9
Getting the source code
Checkout the sources from the Mahout GitHub repositoryeither via
![Apache Maven 3.3 9 Download For Mac Apache Maven 3.3 9 Download For Mac](https://www.programmersought.com/images/576/b615bff1e674abc91860e18df00946d0.png)
Building From Source
Prerequisites:
Linux Environment (preferably Ubuntu 16.04.x) Note: Currently only the JVM-only build will work on a Mac.gcc > 4.xNVIDIA Card (installed with OpenCL drivers alongside usual GPU drivers)
Downloads
Install java 1.7+ in an easily accessible directory (for this example, ~/java/)http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Create a directory ~/apache/ .
Download apache Maven 3.3.9 and un-tar/gunzip to ~/apache/apache-maven-3.3.9/ .https://maven.apache.org/download.cgi
Download and un-tar/gunzip Hadoop 2.4.1 to ~/apache/hadoop-2.4.1/ .https://archive.apache.org/dist/hadoop/common/hadoop-2.4.1/
Download and un-tar/gunzip spark-1.6.3-bin-hadoop2.4 to ~/apache/ .http://spark.apache.org/downloads.htmlChoose release: Spark-1.6.3 (Nov 07 2016)Choose package type: Pre-Built for Hadoop 2.4
Install ViennaCL 1.7.0+If running Ubuntu 16.04+
Otherwise if your distribution’s package manager does not have a viennniacl-dev package >1.7.0, clone it directly into the directory which will be included in when being compiled by Mahout:
Ensure that the OpenCL 1.2+ drivers are installed (packed with most consumer grade NVIDIA drivers). Not sure about higher end cards.
Clone mahout repository into
~/apache
.Configuration
When building mahout for a spark backend, we need four System Environment variables set:
Mahout on Spark regularly uses one more env variable, the IP of the Spark cluster’s master node (usually the node which one would be logged into).
To use 4 local cores (Spark master need not be running)
To use all available local cores (again, Spark master need not be running)
To point to a cluster with spark running:
![Mac Mac](https://images2017.cnblogs.com/blog/421738/201711/421738-20171108180119294-1248588006.png)
Download Maven 3 3 9
We then add these to the path:
Apache Maven 3.0.5 Download
These should be added to the your ~/.bashrc file.
Building Mahout with Apache Maven
From the $MAHOUT_HOME directory we may issue the commands to build each using mvn profiles.
JVM only:
JVM with native OpenMP level 2 and level 3 matrix/vector Multiplication
JVM with native OpenMP and OpenCL for Level 2 and level 3 matrix/vector Multiplication. (GPU errors fall back to OpenMP, currently only a single GPU/node is supported).
Profiles Reference
Apache Maven 3.0.4 Download
Apache Maven encourages users to make use of build profiles for selectively building modules. Free photo printing software for mac.
The command
Is the basic build command. This default will build the following packages.
The following profiles are available for building optional components:
Apache Maven 3.3.9 Download For Mac
Profile | Description | ||||
---|---|---|---|---|---|
all | Build all modules | ||||
apache-release | Used for releasing Apache Mahout. See How To Release for more information. | ||||
flink-batch | Build Community Engine of Mahout for Apache Flink (Batch) | ||||
h2o | Build Community Engine of Mahout for H2o | ||||
mahout-mr | Build Community maintained Mahout for Map Reduce | ||||
viennacl | Build Experimental ViennaCL (GPU) BLAS Pack | ||||
viennacl-omp | Build Experimental ViennaCL-OpenMP (CPU) BLAS Pack | ||||
docs | Profile for building Java/Scala Docs | ||||
Mahout Specific Option | Description | ||||
skipCli | Skip building the command line drivers for Mahout on Apache Spark |