MapReduce 應用程式主控項 REST API 讓使用者取得正在執行的 MapReduce 應用程式主控項的狀態。目前這等同於正在執行的 MapReduce 任務。資訊包括應用程式主控項正在執行的任務以及所有任務詳細資料,例如工作、計數器、組態、嘗試等。應透過代理伺服器存取應用程式主控項。此代理伺服器可組態為在資源管理員或獨立主機上執行。代理伺服器 URL 通常如下所示:http://proxy-http-address:port/proxy/appid
。
MapReduce 應用程式主控項資訊資源提供有關該 MapReduce 應用程式主控項的整體資訊。這包括應用程式 ID、啟動時間、使用者、名稱等。
以下兩個 URI 都會提供 MapReduce 應用程式主控項資訊,來自由 appid 值識別的應用程式 ID。
None
當您提出 MapReduce 應用程式主控項資訊的要求時,資訊會以 info 物件傳回。
項目 | 資料類型 | 說明 |
---|---|---|
appId | 字串 | 應用程式 ID |
startedOn | 長整數 | 應用程式啟動時間(自紀元以來的毫秒數) |
name | 字串 | 應用程式的名稱 |
user | 字串 | 啟動應用程式的使用者的使用者名稱 |
elapsedTime | 長整數 | 自應用程式啟動以來的時間(以毫秒為單位) |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0003/ws/v1/mapreduce/info
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "info" : { "appId" : "application_1326232085508_0003", "startedOn" : 1326238244047, "user" : "user1", "name" : "Sleep job", "elapsedTime" : 32374 } }
XML 回應
HTTP 要求
Accept: application/xml GET http://proxy-http-address:port/proxy/application_1326232085508_0003/ws/v1/mapreduce/info
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 223 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <info> <appId>application_1326232085508_0003</appId> <name>Sleep job</name> <user>user1</user> <startedOn>1326238244047</startedOn> <elapsedTime>32407</elapsedTime> </info>
任務資源提供在這個應用程式主控項上執行的任務清單。另請參閱 任務 API,以取得任務物件的語法。
None
當您提出任務清單的要求時,資訊會以任務物件的集合傳回。另請參閱 任務 API,以取得任務物件的語法。
項目 | 資料類型 | 說明 |
---|---|---|
作業 | 作業物件陣列 (JSON)/零個或多個作業物件 (XML) | 作業物件的集合 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "jobs" : { "job" : [ { "runningReduceAttempts" : 1, "reduceProgress" : 100, "failedReduceAttempts" : 0, "newMapAttempts" : 0, "mapsRunning" : 0, "state" : "RUNNING", "successfulReduceAttempts" : 0, "reducesRunning" : 1, "acls" : [ { "value" : " ", "name" : "mapreduce.job.acl-modify-job" }, { "value" : " ", "name" : "mapreduce.job.acl-view-job" } ], "reducesPending" : 0, "user" : "user1", "reducesTotal" : 1, "mapsCompleted" : 1, "startTime" : 1326238769379, "id" : "job_1326232085508_4_4", "successfulMapAttempts" : 1, "runningMapAttempts" : 0, "newReduceAttempts" : 0, "name" : "Sleep job", "mapsPending" : 0, "elapsedTime" : 59377, "reducesCompleted" : 0, "mapProgress" : 100, "diagnostics" : "", "failedMapAttempts" : 0, "killedReduceAttempts" : 0, "mapsTotal" : 1, "uberized" : false, "killedMapAttempts" : 0, "finishTime" : 0 } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 1214 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobs> <job> <startTime>1326238769379</startTime> <finishTime>0</finishTime> <elapsedTime>59416</elapsedTime> <id>job_1326232085508_4_4</id> <name>Sleep job</name> <user>user1</user> <state>RUNNING</state> <mapsTotal>1</mapsTotal> <mapsCompleted>1</mapsCompleted> <reducesTotal>1</reducesTotal> <reducesCompleted>0</reducesCompleted> <mapProgress>100.0</mapProgress> <reduceProgress>100.0</reduceProgress> <mapsPending>0</mapsPending> <mapsRunning>0</mapsRunning> <reducesPending>0</reducesPending> <reducesRunning>1</reducesRunning> <uberized>false</uberized> <diagnostics/> <newReduceAttempts>0</newReduceAttempts> <runningReduceAttempts>1</runningReduceAttempts> <failedReduceAttempts>0</failedReduceAttempts> <killedReduceAttempts>0</killedReduceAttempts> <successfulReduceAttempts>0</successfulReduceAttempts> <newMapAttempts>0</newMapAttempts> <runningMapAttempts>0</runningMapAttempts> <failedMapAttempts>0</failedMapAttempts> <killedMapAttempts>0</killedMapAttempts> <successfulMapAttempts>1</successfulMapAttempts> <acls> <name>mapreduce.job.acl-modify-job</name> <value> </value> </acls> <acls> <name>mapreduce.job.acl-view-job</name> <value> </value> </acls> </job> </jobs>
作業資源包含由這個應用程式主控程式啟動的特定作業資訊。某些欄位僅在使用者擁有權限時才能存取,視 ACL 設定而定。
使用下列 URI 取得作業物件,針對由 jobid 值識別的作業。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 作業 ID |
name | 字串 | 作業名稱 |
user | 字串 | 使用者名稱 |
狀態 | 字串 | 作業狀態,有效值為:NEW、INITED、RUNNING、SUCCEEDED、FAILED、KILL_WAIT、KILLED、ERROR |
startTime | 長整數 | 作業開始時間 (自紀元以來的毫秒數) |
finishTime | 長整數 | 作業完成時間 (自紀元以來的毫秒數) |
elapsedTime | 長整數 | 自作業開始以來的經過時間 (毫秒數) |
mapsTotal | int | 總共的對應數量 |
mapsCompleted | int | 已完成的對應數量 |
reducesTotal | int | 總共的還原數量 |
reducesCompleted | int | 已完成的還原數量 |
diagnostics | 字串 | 診斷訊息 |
uberized | boolean | 指示作業是否為 uber 作業,完全在應用程式主控程式中執行 |
mapsPending | int | 仍要執行的對應數量 |
mapsRunning | int | 正在執行的對應數量 |
reducesPending | int | 仍要執行的還原數量 |
reducesRunning | int | 正在執行的還原數量 |
newReduceAttempts | int | 新的還原嘗試數量 |
runningReduceAttempts | int | 正在執行的還原嘗試數量 |
failedReduceAttempts | int | 失敗的還原嘗試數量 |
killedReduceAttempts | int | 已終止的還原嘗試數量 |
successfulReduceAttempts | int | 成功的還原嘗試數量 |
newMapAttempts | int | 新的對應嘗試數量 |
runningMapAttempts | int | 正在執行的對應嘗試數量 |
failedMapAttempts | int | 失敗的對應嘗試數量 |
killedMapAttempts | int | 已中斷的映射嘗試次數 |
successfulMapAttempts | int | 成功的映射嘗試次數 |
acls | acl 陣列 (json)/零個或多個 acl 物件 (xml) | acl 物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
value | 字串 | acl 值 |
name | 字串 | acl 名稱 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Server: Jetty(6.1.26) Content-Length: 720
回應本文
{ "job" : { "runningReduceAttempts" : 1, "reduceProgress" : 100, "failedReduceAttempts" : 0, "newMapAttempts" : 0, "mapsRunning" : 0, "state" : "RUNNING", "successfulReduceAttempts" : 0, "reducesRunning" : 1, "acls" : [ { "value" : " ", "name" : "mapreduce.job.acl-modify-job" }, { "value" : " ", "name" : "mapreduce.job.acl-view-job" } ], "reducesPending" : 0, "user" : "user1", "reducesTotal" : 1, "mapsCompleted" : 1, "startTime" : 1326238769379, "id" : "job_1326232085508_4_4", "successfulMapAttempts" : 1, "runningMapAttempts" : 0, "newReduceAttempts" : 0, "name" : "Sleep job", "mapsPending" : 0, "elapsedTime" : 59437, "reducesCompleted" : 0, "mapProgress" : 100, "diagnostics" : "", "failedMapAttempts" : 0, "killedReduceAttempts" : 0, "mapsTotal" : 1, "uberized" : false, "killedMapAttempts" : 0, "finishTime" : 0 } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4 Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 1201 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <job> <startTime>1326238769379</startTime> <finishTime>0</finishTime> <elapsedTime>59474</elapsedTime> <id>job_1326232085508_4_4</id> <name>Sleep job</name> <user>user1</user> <state>RUNNING</state> <mapsTotal>1</mapsTotal> <mapsCompleted>1</mapsCompleted> <reducesTotal>1</reducesTotal> <reducesCompleted>0</reducesCompleted> <mapProgress>100.0</mapProgress> <reduceProgress>100.0</reduceProgress> <mapsPending>0</mapsPending> <mapsRunning>0</mapsRunning> <reducesPending>0</reducesPending> <reducesRunning>1</reducesRunning> <uberized>false</uberized> <diagnostics/> <newReduceAttempts>0</newReduceAttempts> <runningReduceAttempts>1</runningReduceAttempts> <failedReduceAttempts>0</failedReduceAttempts> <killedReduceAttempts>0</killedReduceAttempts> <successfulReduceAttempts>0</successfulReduceAttempts> <newMapAttempts>0</newMapAttempts> <runningMapAttempts>0</runningMapAttempts> <failedMapAttempts>0</failedMapAttempts> <killedMapAttempts>0</killedMapAttempts> <successfulMapAttempts>1</successfulMapAttempts> <acls> <name>mapreduce.job.acl-modify-job</name> <value> </value> </acls> <acls> <name>mapreduce.job.acl-view-job</name> <value> </value> </acls> </job>
使用作業嘗試 API,您可以取得代表作業嘗試的資源集合。當您對此資源執行 GET 作業時,您會取得作業嘗試物件集合。
None
當您要求作業嘗試清單時,資訊會傳回作業嘗試物件陣列。
項目 | 資料類型 | 說明 |
---|---|---|
jobAttempt | 作業嘗試物件陣列 (JSON)/零個或多個作業嘗試物件 (XML) | 作業嘗試物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 作業嘗試 ID |
nodeId | 字串 | 嘗試執行節點的節點 ID |
nodeHttpAddress | 字串 | 嘗試執行節點的節點 HTTP 位址 |
logsLink | 字串 | 作業嘗試記錄的 HTTP 連結 |
containerId | 字串 | 作業嘗試容器的 ID |
startTime | 長整數 | 嘗試開始時間 (自紀元以來的毫秒數) |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "jobAttempts" : { "jobAttempt" : [ { "nodeId" : "host.domain.com:8041", "nodeHttpAddress" : "host.domain.com:8042", "startTime" : 1326238773493, "id" : 1, "logsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001", "containerId" : "container_1326232085508_0004_01_000001" } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 498 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobAttempts> <jobAttempt> <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress> <nodeId>host.domain.com:8041</nodeId> <id>1</id> <startTime>1326238773493</startTime> <containerId>container_1326232085508_0004_01_000001</containerId> <logsLink>http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001</logsLink> </jobAttempt> </jobAttempts>
使用作業計數器 API,您可以取得代表該作業所有計數器的資源集合。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 作業 ID |
counterGroup | 計數器群組物件陣列 (JSON)/零個或多個計數器群組物件 (XML) | 計數器群組物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
counterGroupName | 字串 | 計數器群組名稱 |
counter | 計數器物件陣列 (JSON)/零個或多個計數器物件 (XML) | 計數器物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
name | 字串 | 計數器名稱 |
reduceCounterValue | 長整數 | 縮減作業的計數器值 |
mapCounterValue | 長整數 | 對應作業的計數器值 |
totalCounterValue | 長整數 | 所有作業的計數器值 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/counters
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "jobCounters" : { "id" : "job_1326232085508_4_4", "counterGroup" : [ { "counterGroupName" : "Shuffle Errors", "counter" : [ { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "BAD_ID" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "CONNECTION" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "IO_ERROR" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "WRONG_LENGTH" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "WRONG_MAP" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "WRONG_REDUCE" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter", "counter" : [ { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 2483, "name" : "FILE_BYTES_READ" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 108763, "name" : "FILE_BYTES_WRITTEN" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "FILE_READ_OPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "FILE_LARGE_READ_OPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "FILE_WRITE_OPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 48, "name" : "HDFS_BYTES_READ" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "HDFS_BYTES_WRITTEN" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1, "name" : "HDFS_READ_OPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "HDFS_LARGE_READ_OPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "HDFS_WRITE_OPS" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter", "counter" : [ { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1, "name" : "MAP_INPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1200, "name" : "MAP_OUTPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 4800, "name" : "MAP_OUTPUT_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 2235, "name" : "MAP_OUTPUT_MATERIALIZED_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 48, "name" : "SPLIT_RAW_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "COMBINE_INPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "COMBINE_OUTPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 460, "name" : "REDUCE_INPUT_GROUPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 2235, "name" : "REDUCE_SHUFFLE_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 460, "name" : "REDUCE_INPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "REDUCE_OUTPUT_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1200, "name" : "SPILLED_RECORDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1, "name" : "SHUFFLED_MAPS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "FAILED_SHUFFLE" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1, "name" : "MERGED_MAP_OUTPUTS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 58, "name" : "GC_TIME_MILLIS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 1580, "name" : "CPU_MILLISECONDS" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 462643200, "name" : "PHYSICAL_MEMORY_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 2149728256, "name" : "VIRTUAL_MEMORY_BYTES" }, { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 357957632, "name" : "COMMITTED_HEAP_BYTES" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter", "counter" : [ { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "BYTES_READ" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter", "counter" : [ { "reduceCounterValue" : 0, "mapCounterValue" : 0, "totalCounterValue" : 0, "name" : "BYTES_WRITTEN" } ] } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/counters Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 7027 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobCounters> <id>job_1326232085508_4_4</id> <counterGroup> <counterGroupName>Shuffle Errors</counterGroupName> <counter> <name>BAD_ID</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>CONNECTION</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>IO_ERROR</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>WRONG_LENGTH</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>WRONG_MAP</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>WRONG_REDUCE</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> </counterGroup> <counterGroup> <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName> <counter> <name>FILE_BYTES_READ</name> <totalCounterValue>2483</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>FILE_BYTES_WRITTEN</name> <totalCounterValue>108763</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>FILE_READ_OPS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>FILE_LARGE_READ_OPS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>FILE_WRITE_OPS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>HDFS_BYTES_READ</name> <totalCounterValue>48</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>HDFS_BYTES_WRITTEN</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>HDFS_READ_OPS</name> <totalCounterValue>1</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>HDFS_LARGE_READ_OPS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>HDFS_WRITE_OPS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> </counterGroup> <counterGroup> <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName> <counter> <name>MAP_INPUT_RECORDS</name> <totalCounterValue>1</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>MAP_OUTPUT_RECORDS</name> <totalCounterValue>1200</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>MAP_OUTPUT_BYTES</name> <totalCounterValue>4800</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>MAP_OUTPUT_MATERIALIZED_BYTES</name> <totalCounterValue>2235</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>SPLIT_RAW_BYTES</name> <totalCounterValue>48</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>COMBINE_INPUT_RECORDS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>COMBINE_OUTPUT_RECORDS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>REDUCE_INPUT_GROUPS</name> <totalCounterValue>460</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>REDUCE_SHUFFLE_BYTES</name> <totalCounterValue>2235</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>REDUCE_INPUT_RECORDS</name> <totalCounterValue>460</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>REDUCE_OUTPUT_RECORDS</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>SPILLED_RECORDS</name> <totalCounterValue>1200</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>SHUFFLED_MAPS</name> <totalCounterValue>1</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>FAILED_SHUFFLE</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>MERGED_MAP_OUTPUTS</name> <totalCounterValue>1</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>GC_TIME_MILLIS</name> <totalCounterValue>58</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>CPU_MILLISECONDS</name> <totalCounterValue>1580</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>PHYSICAL_MEMORY_BYTES</name> <totalCounterValue>462643200</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>VIRTUAL_MEMORY_BYTES</name> <totalCounterValue>2149728256</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> <counter> <name>COMMITTED_HEAP_BYTES</name> <totalCounterValue>357957632</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> </counterGroup> <counterGroup> <counterGroupName>org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter</counterGroupName> <counter> <name>BYTES_READ</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> </counterGroup> <counterGroup> <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName> <counter> <name>BYTES_WRITTEN</name> <totalCounterValue>0</totalCounterValue> <mapCounterValue>0</mapCounterValue> <reduceCounterValue>0</reduceCounterValue> </counter> </counterGroup> </jobCounters>
作業設定檔資源包含此作業的作業設定檔資訊。
使用下列 URI 從由 jobid 值識別的作業中取得作業設定檔資訊。
None
項目 | 資料類型 | 說明 |
---|---|---|
路徑 | 字串 | 作業設定檔檔案的路徑 |
屬性 | 設定檔屬性陣列 (JSON)/零個或多個屬性物件 (XML) | 屬性物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
name | 字串 | 設定檔屬性的名稱 |
value | 字串 | 設定檔屬性的值 |
來源 | 字串 | 此設定檔物件的來源位置。如果有多個來源,則會顯示歷史記錄,清單中最後一個為最新來源。 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/conf
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
這是輸出的一個小片段,因為輸出非常大。實際輸出包含作業設定檔檔案中的每個屬性。
{ "conf" : { "path" : "hdfs://host.domain.com:9000/user/user1/.staging/job_1326232085508_0004/job.xml", "property" : [ { "value" : "/home/hadoop/hdfs/data", "name" : "dfs.datanode.data.dir", "source" : ["hdfs-site.xml", "job.xml"] }, { "value" : "org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer", "name" : "hadoop.http.filter.initializers" "source" : ["programmatically", "job.xml"] }, { "value" : "/home/hadoop/tmp", "name" : "mapreduce.cluster.temp.dir" "source" : ["mapred-site.xml"] }, ... ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/conf Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 552 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <conf> <path>hdfs://host.domain.com:9000/user/user1/.staging/job_1326232085508_0004/job.xml</path> <property> <name>dfs.datanode.data.dir</name> <value>/home/hadoop/hdfs/data</value> <source>hdfs-site.xml</source> <source>job.xml</source> </property> <property> <name>hadoop.http.filter.initializers</name> <value>org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer</value> <source>programmatically</source> <source>job.xml</source> </property> <property> <name>mapreduce.cluster.temp.dir</name> <value>/home/hadoop/tmp</value> <source>mapred-site.xml</source> </property> ... </conf>
使用作業 API,您可以取得表示作業所有作業的資源集合。當您對此資源執行 GET 作業時,您會取得作業物件集合。
當您要求作業清單時,資訊將會以作業物件陣列的形式傳回。另請參閱 作業 API 以取得作業物件的語法。
項目 | 資料類型 | 說明 |
---|---|---|
作業 | 作業物件陣列 (JSON)/零個或多個作業物件 (XML) | 作業物件集合 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "tasks" : { "task" : [ { "progress" : 100, "elapsedTime" : 2768, "state" : "SUCCEEDED", "startTime" : 1326238773493, "id" : "task_1326232085508_4_4_m_0", "type" : "MAP", "successfulAttempt" : "attempt_1326232085508_4_4_m_0_0", "finishTime" : 1326238776261 }, { "progress" : 100, "elapsedTime" : 0, "state" : "RUNNING", "startTime" : 1326238777460, "id" : "task_1326232085508_4_4_r_0", "type" : "REDUCE", "successfulAttempt" : "", "finishTime" : 0 } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 603 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <tasks> <task> <startTime>1326238773493</startTime> <finishTime>1326238776261</finishTime> <elapsedTime>2768</elapsedTime> <progress>100.0</progress> <id>task_1326232085508_4_4_m_0</id> <state>SUCCEEDED</state> <type>MAP</type> <successfulAttempt>attempt_1326232085508_4_4_m_0_0</successfulAttempt> </task> <task> <startTime>1326238777460</startTime> <finishTime>0</finishTime> <elapsedTime>0</elapsedTime> <progress>100.0</progress> <id>task_1326232085508_4_4_r_0</id> <state>RUNNING</state> <type>REDUCE</type> <successfulAttempt/> </task> </tasks>
作業資源包含作業中特定作業的資訊。
使用下列 URI 從由 taskid 值識別的作業中取得作業物件。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 工作 ID |
狀態 | 字串 | 工作的狀態 - 有效值為:NEW、SCHEDULED、RUNNING、SUCCEEDED、FAILED、KILL_WAIT、KILLED |
類型 | 字串 | 工作類型 - MAP 或 REDUCE |
successfulAttempt | 字串 | 最後一次成功嘗試的 ID |
進度 | 浮點數 | 工作的進度,以百分比表示 |
startTime | 長整數 | 工作開始的時間(自紀元以來的毫秒數) |
finishTime | 長整數 | 工作結束的時間(自紀元以來的毫秒數) |
elapsedTime | 長整數 | 自應用程式開始以來經過的時間(以毫秒為單位) |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "task" : { "progress" : 100, "elapsedTime" : 0, "state" : "RUNNING", "startTime" : 1326238777460, "id" : "task_1326232085508_4_4_r_0", "type" : "REDUCE", "successfulAttempt" : "", "finishTime" : 0 } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0 Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 299 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <task> <startTime>1326238777460</startTime> <finishTime>0</finishTime> <elapsedTime>0</elapsedTime> <progress>100.0</progress> <id>task_1326232085508_4_4_r_0</id> <state>RUNNING</state> <type>REDUCE</type> <successfulAttempt/> </task>
使用工作計數器 API,您可以物件化一個資源集合,代表該工作的計數器。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 工作 ID |
taskcounterGroup | 計數器群組物件陣列 (JSON)/零個或多個計數器群組物件 (XML) | 計數器群組物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
counterGroupName | 字串 | 計數器群組名稱 |
counter | 計數器物件陣列 (JSON)/零個或多個計數器物件 (XML) | 計數器物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
name | 字串 | 計數器名稱 |
value | 長整數 | 計數器的值 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/counters
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "jobTaskCounters" : { "id" : "task_1326232085508_4_4_r_0", "taskCounterGroup" : [ { "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter", "counter" : [ { "value" : 2363, "name" : "FILE_BYTES_READ" }, { "value" : 54372, "name" : "FILE_BYTES_WRITTEN" }, { "value" : 0, "name" : "FILE_READ_OPS" }, { "value" : 0, "name" : "FILE_LARGE_READ_OPS" }, { "value" : 0, "name" : "FILE_WRITE_OPS" }, { "value" : 0, "name" : "HDFS_BYTES_READ" }, { "value" : 0, "name" : "HDFS_BYTES_WRITTEN" }, { "value" : 0, "name" : "HDFS_READ_OPS" }, { "value" : 0, "name" : "HDFS_LARGE_READ_OPS" }, { "value" : 0, "name" : "HDFS_WRITE_OPS" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter", "counter" : [ { "value" : 0, "name" : "COMBINE_INPUT_RECORDS" }, { "value" : 0, "name" : "COMBINE_OUTPUT_RECORDS" }, { "value" : 460, "name" : "REDUCE_INPUT_GROUPS" }, { "value" : 2235, "name" : "REDUCE_SHUFFLE_BYTES" }, { "value" : 460, "name" : "REDUCE_INPUT_RECORDS" }, { "value" : 0, "name" : "REDUCE_OUTPUT_RECORDS" }, { "value" : 0, "name" : "SPILLED_RECORDS" }, { "value" : 1, "name" : "SHUFFLED_MAPS" }, { "value" : 0, "name" : "FAILED_SHUFFLE" }, { "value" : 1, "name" : "MERGED_MAP_OUTPUTS" }, { "value" : 26, "name" : "GC_TIME_MILLIS" }, { "value" : 860, "name" : "CPU_MILLISECONDS" }, { "value" : 107839488, "name" : "PHYSICAL_MEMORY_BYTES" }, { "value" : 1123147776, "name" : "VIRTUAL_MEMORY_BYTES" }, { "value" : 57475072, "name" : "COMMITTED_HEAP_BYTES" } ] }, { "counterGroupName" : "Shuffle Errors", "counter" : [ { "value" : 0, "name" : "BAD_ID" }, { "value" : 0, "name" : "CONNECTION" }, { "value" : 0, "name" : "IO_ERROR" }, { "value" : 0, "name" : "WRONG_LENGTH" }, { "value" : 0, "name" : "WRONG_MAP" }, { "value" : 0, "name" : "WRONG_REDUCE" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter", "counter" : [ { "value" : 0, "name" : "BYTES_WRITTEN" } ] } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/counters Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 2660 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskCounters> <id>task_1326232085508_4_4_r_0</id> <taskCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName> <counter> <name>FILE_BYTES_READ</name> <value>2363</value> </counter> <counter> <name>FILE_BYTES_WRITTEN</name> <value>54372</value> </counter> <counter> <name>FILE_READ_OPS</name> <value>0</value> </counter> <counter> <name>FILE_LARGE_READ_OPS</name> <value>0</value> </counter> <counter> <name>FILE_WRITE_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_BYTES_READ</name> <value>0</value> </counter> <counter> <name>HDFS_BYTES_WRITTEN</name> <value>0</value> </counter> <counter> <name>HDFS_READ_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_LARGE_READ_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_WRITE_OPS</name> <value>0</value> </counter> </taskCounterGroup> <taskCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName> <counter> <name>COMBINE_INPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>COMBINE_OUTPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>REDUCE_INPUT_GROUPS</name> <value>460</value> </counter> <counter> <name>REDUCE_SHUFFLE_BYTES</name> <value>2235</value> </counter> <counter> <name>REDUCE_INPUT_RECORDS</name> <value>460</value> </counter> <counter> <name>REDUCE_OUTPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>SPILLED_RECORDS</name> <value>0</value> </counter> <counter> <name>SHUFFLED_MAPS</name> <value>1</value> </counter> <counter> <name>FAILED_SHUFFLE</name> <value>0</value> </counter> <counter> <name>MERGED_MAP_OUTPUTS</name> <value>1</value> </counter> <counter> <name>GC_TIME_MILLIS</name> <value>26</value> </counter> <counter> <name>CPU_MILLISECONDS</name> <value>860</value> </counter> <counter> <name>PHYSICAL_MEMORY_BYTES</name> <value>107839488</value> </counter> <counter> <name>VIRTUAL_MEMORY_BYTES</name> <value>1123147776</value> </counter> <counter> <name>COMMITTED_HEAP_BYTES</name> <value>57475072</value> </counter> </taskCounterGroup> <taskCounterGroup> <counterGroupName>Shuffle Errors</counterGroupName> <counter> <name>BAD_ID</name> <value>0</value> </counter> <counter> <name>CONNECTION</name> <value>0</value> </counter> <counter> <name>IO_ERROR</name> <value>0</value> </counter> <counter> <name>WRONG_LENGTH</name> <value>0</value> </counter> <counter> <name>WRONG_MAP</name> <value>0</value> </counter> <counter> <name>WRONG_REDUCE</name> <value>0</value> </counter> </taskCounterGroup> <taskCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName> <counter> <name>BYTES_WRITTEN</name> <value>0</value> </counter> </taskCounterGroup> </jobTaskCounters>
使用工作嘗試 API,您可以取得一個資源集合,代表工作中的工作嘗試。當您對此資源執行 GET 作業時,您會取得一個工作嘗試物件集合。
None
當您要求工作嘗試清單時,資訊會以工作嘗試物件陣列的形式傳回。另請參閱 工作嘗試 API,以取得工作物件的語法。
項目 | 資料類型 | 說明 |
---|---|---|
taskAttempt | 工作嘗試物件陣列 (JSON)/零個或多個工作嘗試物件 (XML) | 工作嘗試物件集合 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "taskAttempts" : { "taskAttempt" : [ { "elapsedMergeTime" : 47, "shuffleFinishTime" : 1326238780052, "assignedContainerId" : "container_1326232085508_0004_01_000003", "progress" : 100, "elapsedTime" : 0, "state" : "RUNNING", "elapsedShuffleTime" : 2592, "mergeFinishTime" : 1326238780099, "rack" : "/98.139.92.0", "elapsedReduceTime" : 0, "nodeHttpAddress" : "host.domain.com:8042", "type" : "REDUCE", "startTime" : 1326238777460, "id" : "attempt_1326232085508_4_4_r_0_0", "finishTime" : 0 } ] } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 807 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <taskAttempts> <taskAttempt> <startTime>1326238777460</startTime> <finishTime>0</finishTime> <elapsedTime>0</elapsedTime> <progress>100.0</progress> <id>attempt_1326232085508_4_4_r_0_0</id> <rack>/98.139.92.0</rack> <state>RUNNING</state> <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress> <type>REDUCE</type> <assignedContainerId>container_1326232085508_0004_01_000003</assignedContainerId> <shuffleFinishTime>1326238780052</shuffleFinishTime> <mergeFinishTime>1326238780099</mergeFinishTime> <elapsedShuffleTime>2592</elapsedShuffleTime> <elapsedMergeTime>47</elapsedMergeTime> <elapsedReduceTime>0</elapsedReduceTime> </taskAttempt> </taskAttempts>
工作嘗試資源包含工作中特定工作嘗試的資訊。
使用下列 URI 從由 attemptid 值識別的工作中取得工作嘗試物件。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 工作 ID |
機架 | 字串 | 機架 |
狀態 | 字串 | 工作嘗試的狀態 - 有效值為:NEW、UNASSIGNED、ASSIGNED、RUNNING、COMMIT_PENDING、SUCCESS_CONTAINER_CLEANUP、SUCCEEDED、FAIL_CONTAINER_CLEANUP、FAIL_TASK_CLEANUP、FAILED、KILL_CONTAINER_CLEANUP、KILL_TASK_CLEANUP、KILLED |
類型 | 字串 | 工作的類型 |
assignedContainerId | 字串 | 此嘗試被指定到的容器 ID |
nodeHttpAddress | 字串 | 此工作嘗試執行的節點的 HTTP 位址 |
diagnostics | 字串 | 診斷訊息 |
進度 | 浮點數 | 工作嘗試的進度,以百分比表示 |
startTime | 長整數 | 工作嘗試開始的時間(自紀元以來的毫秒數) |
finishTime | 長整數 | 工作嘗試結束的時間(自紀元以來的毫秒數) |
elapsedTime | 長整數 | 任務嘗試開始後經過的時間(以毫秒為單位) |
若要減少任務嘗試,您還有下列欄位
項目 | 資料類型 | 說明 |
---|---|---|
shuffleFinishTime | 長整數 | 洗牌完成的時間(自紀元以來的毫秒數) |
mergeFinishTime | 長整數 | 合併完成的時間(自紀元以來的毫秒數) |
elapsedShuffleTime | 長整數 | 洗牌階段完成所需時間(減少任務開始與洗牌完成之間的毫秒數時間) |
elapsedMergeTime | 長整數 | 合併階段完成所需時間(洗牌完成與合併完成之間的毫秒數時間) |
elapsedReduceTime | 長整數 | 減少階段完成所需時間(合併完成與減少任務結束之間的毫秒數時間) |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "taskAttempt" : { "elapsedMergeTime" : 47, "shuffleFinishTime" : 1326238780052, "assignedContainerId" : "container_1326232085508_0004_01_000003", "progress" : 100, "elapsedTime" : 0, "state" : "RUNNING", "elapsedShuffleTime" : 2592, "mergeFinishTime" : 1326238780099, "rack" : "/98.139.92.0", "elapsedReduceTime" : 0, "nodeHttpAddress" : "host.domain.com:8042", "startTime" : 1326238777460, "id" : "attempt_1326232085508_4_4_r_0_0", "type" : "REDUCE", "finishTime" : 0 } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0 Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 691 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <taskAttempt> <startTime>1326238777460</startTime> <finishTime>0</finishTime> <elapsedTime>0</elapsedTime> <progress>100.0</progress> <id>attempt_1326232085508_4_4_r_0_0</id> <rack>/98.139.92.0</rack> <state>RUNNING</state> <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress> <type>REDUCE</type> <assignedContainerId>container_1326232085508_0004_01_000003</assignedContainerId> <shuffleFinishTime>1326238780052</shuffleFinishTime> <mergeFinishTime>1326238780099</mergeFinishTime> <elapsedShuffleTime>2592</elapsedShuffleTime> <elapsedMergeTime>47</elapsedMergeTime> <elapsedReduceTime>0</elapsedReduceTime> </taskAttempt>
使用任務嘗試狀態 API,您可以查詢已提交任務嘗試的狀態,也可以透過修改執行中任務嘗試的狀態,並使用將狀態設定為「已終止」的 PUT 要求,終止執行中的任務嘗試。若要執行 PUT 作業,必須為 AM 網路服務設定驗證。此外,您必須有權限終止任務嘗試。目前您只能將狀態變更為「已終止」;嘗試將狀態變更為任何其他結果,將會產生 400 錯誤回應。未經授權和錯誤要求的錯誤範例如下。當您執行成功的 PUT 時,初始回應可能是 202。您可以重複 PUT 要求,直到您取得 200、使用 GET 方法查詢狀態,或查詢任務嘗試資訊並檢查狀態,以確認應用程式已終止。在下列範例中,我們重複 PUT 要求並取得 200 回應。
請注意,若要終止任務嘗試,您必須為 HTTP 介面設定驗證篩選器。此功能需要在 HttpServletRequest 中設定使用者名稱。如果未設定篩選器,回應將會是「未經授權」回應。
此功能目前處於 alpha 階段,未來可能會變更。
None
當您要求應用程式的狀態時,傳回的資訊有下列欄位
項目 | 資料類型 | 說明 |
---|---|---|
狀態 | 字串 | 應用程式狀態 - 可以是「新增」、「開始」、「執行中」、「提交待處理」、「成功」、「失敗」、「已終止」之一 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Server: Jetty(6.1.26) Content-Length: 20
回應本文
{ "state":"STARTING" }
HTTP 要求
PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
要求主體
{ "state":"KILLED" }
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Server: Jetty(6.1.26) Content-Length: 18
回應本文
{ "state":"KILLED" }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Server: Jetty(6.1.26) Content-Length: 121
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptState> <state>STARTING</state> </jobTaskAttemptState>
HTTP 要求
PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
要求主體
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptState> <state>KILLED</state> </jobTaskAttemptState>
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Server: Jetty(6.1.26) Content-Length: 121
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptState> <state>KILLED</state> </jobTaskAttemptState>
未經授權的錯誤回應
HTTP 要求
PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
要求主體
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptState> <state>KILLED</state> </jobTaskAttemptState>
回應標頭
HTTP/1.1 403 Unauthorized Content-Type: application/json Server: Jetty(6.1.26)
錯誤要求的錯誤回應
HTTP 要求
PUT http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
要求主體
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptState> <state>RUNNING</state> </jobTaskAttemptState>
回應標頭
HTTP/1.1 400 Content-Length: 295 Content-Type: application/xml Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <RemoteException> <exception>BadRequestException</exception> <message>java.lang.Exception: Only 'KILLED' is allowed as a target state.</message> <javaClassName>org.apache.hadoop.yarn.webapp.BadRequestException</javaClassName> </RemoteException>
透過任務嘗試計數器 API,您可以物件化一組資源,這些資源代表該任務嘗試的所有計數器。
None
項目 | 資料類型 | 說明 |
---|---|---|
id | 字串 | 任務嘗試 ID |
taskAttemptcounterGroup | 任務嘗試 counterGroup 物件陣列 (JSON)/零個或多個任務嘗試 counterGroup 物件 (XML) | 一組任務嘗試 counter 群組物件 |
項目 | 資料類型 | 說明 |
---|---|---|
counterGroupName | 字串 | 計數器群組名稱 |
counter | 計數器物件陣列 (JSON)/零個或多個計數器物件 (XML) | 計數器物件集合 |
項目 | 資料類型 | 說明 |
---|---|---|
name | 字串 | 計數器名稱 |
value | 長整數 | 計數器的值 |
JSON 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0/counters
回應標頭
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26)
回應本文
{ "jobTaskAttemptCounters" : { "taskAttemptCounterGroup" : [ { "counterGroupName" : "org.apache.hadoop.mapreduce.FileSystemCounter", "counter" : [ { "value" : 2363, "name" : "FILE_BYTES_READ" }, { "value" : 54372, "name" : "FILE_BYTES_WRITTEN" }, { "value" : 0, "name" : "FILE_READ_OPS" }, { "value" : 0, "name" : "FILE_LARGE_READ_OPS" }, { "value" : 0, "name" : "FILE_WRITE_OPS" }, { "value" : 0, "name" : "HDFS_BYTES_READ" }, { "value" : 0, "name" : "HDFS_BYTES_WRITTEN" }, { "value" : 0, "name" : "HDFS_READ_OPS" }, { "value" : 0, "name" : "HDFS_LARGE_READ_OPS" }, { "value" : 0, "name" : "HDFS_WRITE_OPS" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.TaskCounter", "counter" : [ { "value" : 0, "name" : "COMBINE_INPUT_RECORDS" }, { "value" : 0, "name" : "COMBINE_OUTPUT_RECORDS" }, { "value" : 460, "name" : "REDUCE_INPUT_GROUPS" }, { "value" : 2235, "name" : "REDUCE_SHUFFLE_BYTES" }, { "value" : 460, "name" : "REDUCE_INPUT_RECORDS" }, { "value" : 0, "name" : "REDUCE_OUTPUT_RECORDS" }, { "value" : 0, "name" : "SPILLED_RECORDS" }, { "value" : 1, "name" : "SHUFFLED_MAPS" }, { "value" : 0, "name" : "FAILED_SHUFFLE" }, { "value" : 1, "name" : "MERGED_MAP_OUTPUTS" }, { "value" : 26, "name" : "GC_TIME_MILLIS" }, { "value" : 860, "name" : "CPU_MILLISECONDS" }, { "value" : 107839488, "name" : "PHYSICAL_MEMORY_BYTES" }, { "value" : 1123147776, "name" : "VIRTUAL_MEMORY_BYTES" }, { "value" : 57475072, "name" : "COMMITTED_HEAP_BYTES" } ] }, { "counterGroupName" : "Shuffle Errors", "counter" : [ { "value" : 0, "name" : "BAD_ID" }, { "value" : 0, "name" : "CONNECTION" }, { "value" : 0, "name" : "IO_ERROR" }, { "value" : 0, "name" : "WRONG_LENGTH" }, { "value" : 0, "name" : "WRONG_MAP" }, { "value" : 0, "name" : "WRONG_REDUCE" } ] }, { "counterGroupName" : "org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter", "counter" : [ { "value" : 0, "name" : "BYTES_WRITTEN" } ] } ], "id" : "attempt_1326232085508_4_4_r_0_0" } }
XML 回應
HTTP 要求
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0/attempts/attempt_1326232085508_4_4_r_0_0/counters Accept: application/xml
回應標頭
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 2735 Server: Jetty(6.1.26)
回應本文
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <jobTaskAttemptCounters> <id>attempt_1326232085508_4_4_r_0_0</id> <taskAttemptCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.FileSystemCounter</counterGroupName> <counter> <name>FILE_BYTES_READ</name> <value>2363</value> </counter> <counter> <name>FILE_BYTES_WRITTEN</name> <value>54372</value> </counter> <counter> <name>FILE_READ_OPS</name> <value>0</value> </counter> <counter> <name>FILE_LARGE_READ_OPS</name> <value>0</value> </counter> <counter> <name>FILE_WRITE_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_BYTES_READ</name> <value>0</value> </counter> <counter> <name>HDFS_BYTES_WRITTEN</name> <value>0</value> </counter> <counter> <name>HDFS_READ_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_LARGE_READ_OPS</name> <value>0</value> </counter> <counter> <name>HDFS_WRITE_OPS</name> <value>0</value> </counter> </taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.TaskCounter</counterGroupName> <counter> <name>COMBINE_INPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>COMBINE_OUTPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>REDUCE_INPUT_GROUPS</name> <value>460</value> </counter> <counter> <name>REDUCE_SHUFFLE_BYTES</name> <value>2235</value> </counter> <counter> <name>REDUCE_INPUT_RECORDS</name> <value>460</value> </counter> <counter> <name>REDUCE_OUTPUT_RECORDS</name> <value>0</value> </counter> <counter> <name>SPILLED_RECORDS</name> <value>0</value> </counter> <counter> <name>SHUFFLED_MAPS</name> <value>1</value> </counter> <counter> <name>FAILED_SHUFFLE</name> <value>0</value> </counter> <counter> <name>MERGED_MAP_OUTPUTS</name> <value>1</value> </counter> <counter> <name>GC_TIME_MILLIS</name> <value>26</value> </counter> <counter> <name>CPU_MILLISECONDS</name> <value>860</value> </counter> <counter> <name>PHYSICAL_MEMORY_BYTES</name> <value>107839488</value> </counter> <counter> <name>VIRTUAL_MEMORY_BYTES</name> <value>1123147776</value> </counter> <counter> <name>COMMITTED_HEAP_BYTES</name> <value>57475072</value> </counter> </taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName>Shuffle Errors</counterGroupName> <counter> <name>BAD_ID</name> <value>0</value> </counter> <counter> <name>CONNECTION</name> <value>0</value> </counter> <counter> <name>IO_ERROR</name> <value>0</value> </counter> <counter> <name>WRONG_LENGTH</name> <value>0</value> </counter> <counter> <name>WRONG_MAP</name> <value>0</value> </counter> <counter> <name>WRONG_REDUCE</name> <value>0</value> </counter> </taskAttemptCounterGroup> <taskAttemptCounterGroup> <counterGroupName>org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter</counterGroupName> <counter> <name>BYTES_WRITTEN</name> <value>0</value> </counter> </taskAttemptCounterGroup> </jobTaskAttemptCounters>