WebHCat (Templeton) Manual

Apache Hive : WebHCat This is the manual for WebHCat, previously known as Templeton. WebHCat is the REST API for HCatalog, a table and storage management layer for Hadoop.  Using WebHCat Installation Configuration Reference See the HCatalog Manual for general HCatalog documentation. Navigation Links Next: Using WebHCat

Apache Hive : WebHCat Configure Apache Hive : WebHCat Configure Configuration Files Configuration Variables Configuration Files The configuration for WebHCat (Templeton) merges the normal Hadoop configuration with the WebHCat-specific variables. Because WebHCat is designed to connect services that are not normally connected, the configuration is more complex than might be desirable. The WebHCat-specific configuration is split into two layers:

Apache Hive : WebHCat Installation Apache Hive : WebHCat Installation WebHCat Installed with Hive WebHCat Installation Procedure Server Commands Requirements Hadoop Distributed Cache Permissions Secure Cluster Proxy User Support WebHCat Installed with Hive WebHCat and HCatalog are installed with Hive, starting with Hive release 0.11.0. If you install Hive from the binary tarball, the WebHCat server command webhcat_server.sh is in the hcatalog/sbin directory.

Apache Hive : WebHCat Reference Reference: WebHCat Resources This overview page lists all of the WebHCat resources. (DDL resources are listed here and on another overview page. For information about HCatalog DDL commands, see HCatalog DDL. For information about Hive DDL commands, see Hive Data Definition Language.)   Category Resource (Type) Description General :version (GET) Return a list of supported response types.   status (GET) Return the WebHCat server status.

Apache Hive : WebHCat Reference AllDDL WebHCat Reference: DDL Resources This is an overview page for the WebHCat DDL resources. The full list of WebHCat resources is on this overview page. For information about HCatalog DDL commands, see HCatalog DDL. For information about Hive DDL commands, see Hive Data Definition Language. Object Resource (Type) Description DDL Command ddl (POST) Perform an HCatalog DDL command.

Apache Hive : WebHCat Reference DDL Apache Hive : WebHCat Reference DDL Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Performs an HCatalog DDL command. The command is executed immediately upon request. Responses are limited to 1 MB. For requests which may return longer results consider using the Hive resource as an alternative. URL http://www.myserver.com/templeton/ddl

Apache Hive : WebHCat Reference DeleteDB Apache Hive : WebHCat Reference DeleteDB Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Delete a database. URL http://www.myserver.com/templeton/v1/ddl/database/:db Parameters Name Description Required? Default :db The database name Required None ifExists Hive returns an error if the database specified does not exist, unless ifExists is set to true.

Apache Hive : WebHCat Reference DeleteJob Apache Hive : WebHCat Reference DeleteJob Description URL Parameters Results Example Curl Command JSON Output Description Kill a job given its job ID. Substitute “:jobid” with the job ID received when the job was created. Version: Deprecated in 0.12.0 DELETE queue/:jobid is deprecated starting in Hive release 0.12.0. Users are encouraged to use DELETE jobs/:jobid instead.

Apache Hive : WebHCat Reference DeleteJobID Apache Hive : WebHCat Reference DeleteJobID Description URL Parameters Results Example Curl Command JSON Output Description Kill a job given its job ID. Substitute “:jobid” with the job ID received when the job was created. Version: Hive 0.12.0 and later DELETE jobs/:jobid is introduced in Hive release 0.12.0. It is equivalent to [DELETE queue/:jobid](https://simhadri-g.github.io/hive-site/docs/latest/webhcat/webhcat-reference-deletejob_34017204/) in prior releases.

Apache Hive : WebHCat Reference DeletePartition Apache Hive : WebHCat Reference DeletePartition Description URL Parameters Results Example Curl Command JSON Output Description Delete (drop) a partition in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/partition/:partition Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :partition The partition name, col_name=‘value’ list.

Apache Hive : WebHCat Reference DeleteTable Apache Hive : WebHCat Reference DeleteTable Description URL Parameters Results Example Curl Command JSON Output Description Delete (drop) an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table Parameters Name Description Required? Default :db The database name Required None :table The table name Required None ifExists Hive 0.70 and later returns an error if the table specified does not exist, unless ifExists is set to true.

Apache Hive : WebHCat Reference GetColumn Apache Hive : WebHCat Reference GetColumn Description URL Parameters Results Example Curl Command JSON Output Description Describe a single column in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/column/:column Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :column The column name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetColumns Apache Hive : WebHCat Reference GetColumns Description URL Parameters Results Example Curl Command JSON Output Description List the columns in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/column Parameters Name Description Required? Default :db The database name Required None :table The table name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetDB Apache Hive : WebHCat Reference GetDB Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Describe a database. (Note: This resource has a “format=extended” parameter however the output structure does not change if it is used.) URL http://www.myserver.com/templeton/v1/ddl/database/:db Parameters Name Description Required? Default :db The database name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetDBs Apache Hive : WebHCat Reference GetDBs Description URL Parameters Results Example Curl Command JSON Output Description List the databases in HCatalog. URL http://www.myserver.com/templeton/v1/ddl/database Parameters Name Description Required? Default like List only databases whose names match the specified pattern. Optional “*” (List all) The standard parameters are also supported.

Apache Hive : WebHCat Reference GetPartition Apache Hive : WebHCat Reference GetPartition Description URL Parameters Results Example Curl Command JSON Output Description Describe a single partition in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/partition/:partition Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :partition The partition name, col_name=‘value’ list.

Apache Hive : WebHCat Reference GetPartitions Apache Hive : WebHCat Reference GetPartitions Description URL Parameters Results Example Curl Command JSON Output Description List all the partitions in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/partition Parameters Name Description Required? Default :db The database name Required None :table The table name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetProperties Apache Hive : WebHCat Reference GetProperties Description URL Parameters Results Example Curl Command JSON Output Description List all the properties of an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/property Parameters Name Description Required? Default :db The database name Required None :table The table name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetProperty Apache Hive : WebHCat Reference GetProperty Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Return the value of a single table property. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/property/:property Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :property The property name Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference GetTable Apache Hive : WebHCat Reference GetTable Description URL Parameters Results Example Curl Command (simple) JSON Output (simple) Curl Command (extended) JSON Output (extended) JSON Output (error) Description Describe an HCatalog table. Normally returns a simple list of columns (using “desc table”), but the extended format will show more information (using “show table extended like”).

Apache Hive : WebHCat Reference GetTables Apache Hive : WebHCat Reference GetTables Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description List the tables in an HCatalog database. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table Parameters Name Description Required? Default :db The database name Required None like List only tables whose names match the specified pattern Optional “*” (List all tables) The standard parameters are also supported.

Apache Hive : WebHCat Reference Hive Apache Hive : WebHCat Reference Hive Description URL Parameters Results Example Curl Command JSON Output Example Results Description Runs a Hive query or set of commands. Version: Hive 0.13.0 and later As of Hive 0.13.0, GET version/hive displays the Hive version used for the query or commands. URL http://www.myserver.com/templeton/v1/hive Parameters Name Description Required?

Apache Hive : WebHCat Reference Job Apache Hive : WebHCat Reference Job Description URL Parameters Results Example Curl Command JSON Output Description Check the status of a job and get related job information given its job ID. Substitute “:jobid” with the job ID received when the job was created. Version: Hive 0.12.0 and later GET jobs/:jobid is introduced in Hive release 0.

Apache Hive : WebHCat Reference JobIDs Apache Hive : WebHCat Reference JobIDs Description URL Parameters Results Example Curl Command JSON Output Description Return a list of all job IDs. Version: Deprecated in 0.12.0 GET queue is deprecated starting in Hive release 0.12.0. (See HIVE-4443.) Users are encouraged to use [GET jobs](https://simhadri-g.github.io/hive-site/docs/latest/webhcat/webhcat-reference-jobs_34835057/) instead. Version: Obsolete in 0.14.0 GET queue will be removed in Hive release 0.

Apache Hive : WebHCat Reference JobInfo Apache Hive : WebHCat Reference JobInfo Description URL Parameters Results Example Curl Command JSON Output JSON Output (Hive 0.12.0 and later) Description Check the status of a job and get related job information given its job ID. Substitute “:jobid” with the job ID received when the job was created. Version: Deprecated in 0.12.0 GET queue/:jobid is deprecated starting in Hive release 0.

Apache Hive : WebHCat Reference Jobs Apache Hive : WebHCat Reference Jobs Description URL Parameters Results Examples Curl Command JSON Output Curl Command (showall) JSON Output (showall) Curl Command (fields) JSON Output (fields) Description Return a list of all job IDs. Version: Hive 0.12.0 and later GET jobs is introduced in Hive release 0.12.0. It is equivalent to [GET queue](https://simhadri-g.github.io/hive-site/docs/latest/webhcat/webhcat-reference-jobids_34017187/) in prior releases.

Apache Hive : WebHCat Reference MapReduceJar Apache Hive : WebHCat Reference MapReduceJar Description URL Parameters Results Example Code and Data Setup Curl Command JSON Output Description Creates and queues a standard Hadoop MapReduce job. Version: Hive 0.13.0 and later As of Hive 0.13.0, GET version/hadoop displays the Hadoop version used for the MapReduce job. URL http://www.myserver.com/templeton/v1/mapreduce/jar Parameters Name Description Required?

Apache Hive : WebHCat Reference MapReduceStream Apache Hive : WebHCat Reference MapReduceStream Description URL Parameters Results Example Code and Data Setup Curl Command JSON Output Example Results Description Create and queue a Hadoop streaming MapReduce job. Version: Hive 0.13.0 and later As of Hive 0.13.0, GET version/hadoop displays the Hadoop version used for the MapReduce job. URL http://www.myserver.com/templeton/v1/mapreduce/streaming Parameters Name Description Required?

Apache Hive : WebHCat Reference Pig Apache Hive : WebHCat Reference Pig Description URL Parameters Results Example Code and Data Setup Curl Command JSON Output Description Create and queue a Pig job. URL http://www.myserver.com/templeton/v1/pig Parameters Name Description Required? Default execute String containing an entire, short Pig program to run. One of either “execute” or “file” is required.

Apache Hive : WebHCat Reference PostTable Apache Hive : WebHCat Reference PostTable Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Rename an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table Parameters Name Description Required? Default :db The database name Required None :table The existing (old) table name Required None rename The new table name Required None group The user group to use Optional None permissions The permissions string to use.

Apache Hive : WebHCat Reference PutColumn Apache Hive : WebHCat Reference PutColumn Description URL Parameters Results Example Curl Command JSON Output Description Create a column in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/column/:column Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :column The column name Required None group The user group to use Optional None permissions The permissions string to use Optional None type The type of column to add, like “string” or “int” Required None comment The column comment, like a description Optional None The standard parameters are also supported.

Apache Hive : WebHCat Reference PutDB Apache Hive : WebHCat Reference PutDB Description URL Parameters Results Example Curl Command JSON Output Description Create a database. URL http://www.myserver.com/templeton/v1/ddl/database/:db Parameters Name Description Required? Default :db The database name Required None group The user group to use Optional None permissions The permissions string to use Optional None location The database location Optional None comment A comment for the database, like a description Optional None properties The database properties Optional None The standard parameters are also supported.

Apache Hive : WebHCat Reference PutPartition Apache Hive : WebHCat Reference PutPartition Description URL Parameters Results Example Curl Command JSON Output Description Create a partition in an HCatalog table. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/partition/:partition Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :partition The partition name, col_name=‘value’ list.

Apache Hive : WebHCat Reference PutProperty Apache Hive : WebHCat Reference PutProperty Description URL Parameters Results Example Curl Command JSON Output Description Add a single property on an HCatalog table. This will also reset an existing property. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table/property/:property Parameters Name Description Required? Default :db The database name Required None :table The table name Required None :property The property name Required None group The user group to use Optional None permissions The permissions string to use Optional None value The property value Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference PutTable Apache Hive : WebHCat Reference PutTable Description URL Parameters Results Example Curl Command Curl Command (using clusteredBy) JSON Output JSON Output (error) Description Create a new HCatalog table. For more information, please refer to the Hive documentation for CREATE TABLE. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:table Parameters Name Description Required? Default :db The database name.

Apache Hive : WebHCat Reference PutTableLike Apache Hive : WebHCat Reference PutTableLike Description URL Parameters Results Example Curl Command JSON Output Description Create a new HCatalog table like an existing one. URL http://www.myserver.com/templeton/v1/ddl/database/:db/table/:existingtable/like/:newtable Parameters Name Description Required? Default :db The database name Required None :existingtable The existing table name Required None :newtable The new table name Required None group The user group to use when creating a table Optional None permissions The permissions string to use when creating a table Optional None external Allows you to specify a location so that Hive does not use the default location for this table.

Apache Hive : WebHCat Reference ResponseTypes Apache Hive : WebHCat Reference ResponseTypes Description URL Parameters Results Example Curl Command JSON Output JSON Output (error) Description Returns a list of the response types supported by WebHCat (Templeton). URL http://www.myserver.com/templeton/:version Parameters Name Description Required? Default :version The WebHCat version number. (Currently this must be “v1”.) Required None The standard parameters are also supported.

Apache Hive : WebHCat Reference Status Apache Hive : WebHCat Reference Status Description URL Parameters Results Example Curl Command JSON Output Description Returns the current status of the WebHCat (Templeton) server. Useful for heartbeat monitoring. URL http://www.myserver.com/templeton/v1/status Parameters Only the standard parameters are accepted. Results Name Description status “ok” if the WebHCat server was contacted.

Apache Hive : WebHCat Reference Version Apache Hive : WebHCat Reference Version Description URL Parameters Results Example Curl Command JSON Output Description Returns a list of supported versions and the current version. URL http://www.myserver.com/templeton/v1/version Parameters Only the standard parameters are accepted. Results Name Description supportedVersions A list of all supported versions. version The current version.

Apache Hive : WebHCat Reference VersionHadoop Apache Hive : WebHCat Reference VersionHadoop Description URL Parameters Results Example Curl Command JSON Output Description Return the version of Hadoop being run when WebHCat creates a MapReduce job (POST mapreduce/jar or mapreduce/streaming). Version: Hive 0.13.0 and later GET version/hadoop is introduced in Hive release 0.13.0 (HIVE-6226). URL http://www.myserver.com/templeton/v1/version/hadoop Parameters Only the standard parameters are accepted.

Apache Hive : WebHCat Reference VersionHive Apache Hive : WebHCat Reference VersionHive Description URL Parameters Results Example Curl Command JSON Output Description Return the version of Hive being run when WebHCat issues Hive queries or commands (POST hive). Version: Hive 0.13.0 and later GET version/hive is introduced in Hive release 0.13.0 (HIVE-6226). URL http://www.myserver.com/templeton/v1/version/hive Parameters Only the standard parameters are accepted.

Apache Hive : WebHCat UsingWebHCat Apache Hive : WebHCat UsingWebHCat Version information Introduction to WebHCat URL Format Security Standard Parameters Security Error Response WebHDFS and Code Push Error Codes and Responses Log Files Project Name Attachments: Version information The HCatalog project graduated from the Apache incubator and merged with the Hive project on March 26, 2013. Hive version 0.11.0 is the first release that includes HCatalog and its REST API, WebHCat.