http://geek.kyloo.net/software/ 2010-05-26T09:38:06-06:00 http://geek.kyloo.net/software/ http://geek.kyloo.net/software/lib/images/favicon.ico text/html 2009-11-28T11:41:06-06:00 http://geek.kyloo.net/software/doku.php/chaski:configure?rev=1259433666&do=diff Chaski as a Jar Chaski provides several useful scripts to make training easy, these will be described later. Leaving the scripts aside, Chaski is packed into a single jar file. It can only be executed with Hadoop version 0.20.1+. * log4j.properties the configure file for java logging system * pkglist.txt list available actions of Chaski and the entry points. text/html 2010-03-08T08:29:29-06:00 http://geek.kyloo.net/software/doku.php/chaski:download?rev=1268062169&do=diff Dependencies Chaski must be run on Hadoop, however it can also be run on local machines with Pseudo-cluster hadoop installations. Most recent Chaski make use of 0.20.1 API, and DOES NOT support any previous version of Hadoop. Chaski package usually contain all its dependencies jar, and after building the package should be able to run directly on Hadoop, i.e. you do not need to add third party jars to the CLASSPATH, all the jar-files not included in Hadoop distribution will be unjar-ed and re-… text/html 2009-11-28T11:40:29-06:00 http://geek.kyloo.net/software/doku.php/chaski:faq?rev=1259433629&do=diff text/html 2009-11-28T11:39:18-06:00 http://geek.kyloo.net/software/doku.php/chaski:install?rev=1259433558&do=diff The page describes the installation of Chaski. 1. Download latest release Please go to download and download latest release. If you only want to run phrase extraction using Chaski, you can just proceed, if you also want to run word alignment distributedly, you need to download and compile MGIZA package. download. text/html 2009-11-28T11:41:29-06:00 http://geek.kyloo.net/software/doku.php/chaski:output_structure?rev=1259433689&do=diff This section will describe typical output on HDFS, so you know where to find what, what can be deleted and what need to be saved. The structuer tree under $ROOT directory is printed in tree-view in the following file. All the directories names in italic format are temporary files and can be deleted safely after training. text/html 2009-11-28T11:37:05-06:00 http://geek.kyloo.net/software/doku.php/chaski:overview?rev=1259433425&do=diff Chaski is a distributed toolkit for machine translation. It contains the following tools: * Distributed word clustering. Being able to build word classes for billion-word corpus. * Distributed word alignment. Using the newest version of overview, it is able to training word alignment models on the cluster in hours instead of days. * Distributed phrase extraction. The phrase extraction for large corpus turns turns out to be slow and require huge disk space and (actually, or) memory, the … text/html 2010-03-08T08:31:20-06:00 http://geek.kyloo.net/software/doku.php/chaski:release_note_chaski?rev=1268062280&do=diff 0.2.5 Maintenance release, fixed bugs when detecting existence of files on HDFS. Download on Source Forge 0.2.4 Maintenance release, fixed bugs that may affect interaction with LoonyBin. Download on Source Forge 0.2.3 Maintenance release, fixed bugs: text/html 2009-11-28T12:01:09-06:00 http://geek.kyloo.net/software/doku.php/chaski:tutorial?rev=1259434869&do=diff The most common usage of Chaski include word alignment and phrase extraction. This tutorial will cover both. 0. Prepare Chaski Download Chaski here. Follow the instruction here to install chaski. And the following part of the tutorial will explain two alternative pipelines: Running full training from word alignment and running only the phrase extraction.