M E S OS F RA M E W OR K AP I V1
@vi n od ko n e
M E SO S A P I S
S che d u l e r M as t e r Ag e n t Exe cu to r
Scheduler API Executor APIInternal API
Operator API
Framework API
N E T W O R K I N G C O N ST RA I N T S
S che d u l e r M a s te r
Firewall
Containers :( Network partitions :(
DE P E NDE NC E O N NAT IVE L IB RA RY
Sch e d u l e r
M a s te rJ a v a L i b r a r y
Na t i v e L i b r a ry
mesos.jar libmesos.so Hard to debugNot portable
Un-needed dependencies
UND O CU M E NT ED F RAM EW OR K AP I
POST /master/mesos.internal.LaunchTasksMessage HTTP/1.1
User-Agent: libprocess/[email protected]:8081
Libprocess-From: [email protected]:8081
Connection: Keep-Alive
Host: Transfer-Encoding: chunked
LIBPROCESS_IP
LIBPROCESS_ADVERTISE_IP
L AC K OF A P I V E R S I O N I N G
S ch e du l e r M a s te r
Version YDriver version XCompatible?
H A R D T O E VO LV E TH E A P I
Lot of boiler plate to expose new API objects
Adding new features breaks the APIMaintenance API?
– HA RVE Y DEN T
“You either die a hero…or live long enough to see yourself become the
villain” replaced by a better API
F RA M E W O R K A P I V 1 G OA L S
API versioning
Easy to evolve API
Well documented
Allow pure language client libraries
Work inside firewalls / containers
F RA M E W O R K A P I V 1
• Calls and Events as building blocks
• Schema: v1/scheduler.proto and v1/executor.proto
• Versioned API endpoints
• master: /api/v1/scheduler
• agent: /api/v1/executor
CAL L S Old APISU BS CR I B E start()TEAR DOWN stop()
ACCE PT acceptOffers()DE CL I NE declineOffer()R EV I VE reviveOffers()
K I LL killTask()SH U TDOWN * Shutdown executor *
ACK NOWL EDG E acknowledgeStatusUpdate()
R ECONC I LE reconcileTasks()MESS AG E sendFrameworkMessage(
)R EQU ES T requestResources()SU P P R ES S suppressOffers()
EVENT S Old API
SU BSC R I B ED registered() / reregistered()
OF FE R S resourceOffers()R ES C I ND offerRescinded()U P DATE statusUpdate()
MESS AG E frameworkMessage()
FA I LU R E slaveLost() / executorLost()
ER R OR error()H EARTB E AT * Periodic heartbeats *
PR O T O C O L
• HTTP 1.1
• Scheduler/Executor opens connections to the master/agent
• A persistent connection to receive Events
• One (or more) connection(s) to send Calls
PR O T O C O L
• Every call is a HTTP POST request
• Content-Type: application/json or application/x-protobuf
• SUBSCRIBE call results in a “200 OK” streaming response
• Record-IO formatted events
• “Mesos-Stream-Id” header
• All successful non-SUBSCRIBE calls result in “202 Accepted”
• Must include “Mesos-Stream-Id" header
SU BS C R I PT I O N R E Q U E ST
POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050Content-Type: application/jsonAccept: application/jsonConnection: close
{ “type” : “SUBSCRIBE”,
“subscribe” : { “framework_info” : { “user” : “foo”, “name” : “Example HTTP Framework” }, }}
SU BS C R I PT I O N R E S PO N SE
HTTP/1.1 200 OK
Content-Type: application/jsonTransfer-Encoding: chunkedMesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
<event length>{ “type” : “SUBSCRIBED”, “subscribed” : { “framework_id” : {“value”:“12220-3440-12532-2345”}, “heartbeat_interval_seconds” : 15 } }<more events>
K I L LPOST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050Content-Type: application/sonAccept: application/jsonConnection: closeMesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{ “framework_id” : {“value” : “12220-3440-12532-2345”},
“type” : “KILL”,
“kill” : { “task_id” : {“value” : “12220-3440-12532-my-task”}, “agent_id” : {“value” : “12220-3440-12532-S123345”} }}
Response: HTTP/1.1 202 Accepted
DI S C O N N E C T I O N S & PA RT I T I O N S• Master tracks the persistent subscription
connection
• Scheduler can reconnect within failover timeout
• Assigns stream IDs to uniquely identify streams
• Periodic HEARTBEATs sent by master
• Scheduler can reconnect if no HEARTBEATs
V E R S I ON I N G
Explicit Simple
Avoid version explosion
/api/v1/scheduler/api/v1/executor
AP I VE R S I O N VS R E L E AS E VER S I O N• API version == Major release version
• v1 API supported by 1.0.0, 1.4.0, 1.20.0
• vN API released in N-1 release version
• vN API considered stable in the last N-1 release
• Version bumping
• Major/API version bumped for backwards incompatible changes (> yearly)
• Minor version bumped regularly (4-8 weeks)
SC H E D U L E R A P I V 1 STATU S• 0.28.0: More API changes (Use this version!)
• 0.24.0: MVP
• Experimental: Use in testing clusters only
• Schema: v1/scheduler/scheduler.proto
• Documentation: http://mesos.apache.org/documentation/latest/scheduler-http-api/
• Reference implementation
• C++ library: https://github.com/apache/mesos/blob/master/src/scheduler/scheduler.cpp
• C++ scheduler: https://github.com/apache/mesos/blob/master/src/examples/test_http_framework.cpp
E XE C U T O R A P I V 1 S TATU S
• 0.28.0: MVP
• Experimental: Use in testing clusters only
• Schema: v1/executor/executor.proto
• Documentation: http://mesos.apache.org/documentation/latest/executor-http-api/
• Reference implementation
• C++ library: https://github.com/apache/mesos/blob/master/src/executor/executor.cpp
• C++ executor: https://github.com/apache/mesos/blob/master/src/examples/test_http_executor.cpp
TO D O
• Authentication support
• Encryption support
• Rate limiting support
• Scheduler <—> Executor message optimization
T H A N K YO U
Questions?
Top Related