Tour of Restate
- TypeScript
- Java
- Go
- Python
- Rust
This tutorial guides you through the development of an end-to-end Restate application, and covers all the essential features. After this tutorial, you should have a firm understanding of how Restate can help you and feel comfortable to tackle your next application on your own.
This tutorial implements a ticket reservation application for a theatre. It allows users to add tickets for specific seats to their shopping cart. After the user adds a ticket, the seat gets reserved for 15 minutes. If the user doesn't pay for the ticket within that time interval, the reservation is released and the ticket becomes available to other users.
Restate applications are made up of services that expose handlers. Handlers are functions that implement business logic. Restate manages their invocation and execution. Services communicate with one another using Remote Procedure Calls (RPC). Our ticket example consists of three services:
As we go, you will discover how Restate can help you with some intricacies in this application.
- TypeScript
- Java
- Go
- Python
- Rust
- Latest stable version of NodeJS >= v18.17.1 and npm CLI >= 9.6.7 installed.
- Install Restate Server and CLI
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- TypeScript SDK version:
1.5.3
- Restate Server Docker image:
docker.restate.dev/restatedev/restate:1.3
- JDK >= 17
- Install Restate Server and CLI
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Java SDK version:
2.0.0
- Restate Server Docker image:
docker.restate.dev/restatedev/restate:1.3
- Go >= 1.21.0
- Install Restate Server and CLI
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Go SDK version:
0.4.0
- Restate Server Docker image:
docker.restate.dev/restatedev/restate:1.3
- Python >= 3.11
- Install Restate Server and CLI
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Python SDK version:
0.6.0
- Restate Server Docker image:
docker.restate.dev/restatedev/restate:1.3
- Rust
- Install Restate Server and CLI
- Optional: Docker Engine or Podman, if you want to run the Restate Server with Docker. And to run Jaeger.
This guide is written for:
- Rust SDK version: 0.4.0
- Restate Server Docker image:
docker.restate.dev/restatedev/restate:1.3
Getting Started
Set up the services
- TypeScript
- Java
- Go
- Python
- Rust
Download the example and run locally with an IDE:
- CLI
- wget
restate example typescript-tour-of-restate && cd typescript-tour-of-restate
wget https://github.com/restatedev/examples/releases/latest/download/typescript-tour-of-restate.zip &&unzip typescript-tour-of-restate.zip -d typescript-tour-of-restate &&rm typescript-tour-of-restate.zip
Install the dependencies and build the app:
npm install && npm run build
Run the services
npm run app-dev
This GitHub repository contains the basic skeleton of the TypeScript services that you develop in this tutorial.
- CLI
- wget
restate example java-tour-of-restate && cd java-tour-of-restate
wget https://github.com/restatedev/examples/releases/latest/download/java-tour-of-restate.zip &&unzip java-tour-of-restate.zip -d java-tour-of-restate &&rm java-tour-of-restate.zip && cd java-tour-of-restate
Run the services
./gradlew run
This GitHub repository contains the basic skeleton of the Java services that you develop in this tutorial.
- CLI
- wget
restate example go-tour-of-restate && cd go-tour-of-restate
wget https://github.com/restatedev/examples/releases/latest/download/go-tour-of-restate.zip &&unzip go-tour-of-restate.zip -d go-tour-of-restate &&rm go-tour-of-restate.zip && cd go-tour-of-restate
Run the services
go run ./app
Download the example and run locally with an IDE:
- CLI
- wget
restate example python-tour-of-restate && cd python-tour-of-restate
wget https://github.com/restatedev/examples/releases/latest/download/python-tour-of-restate.zip &&unzip python-tour-of-restate.zip -d python-tour-of-restate &&rm python-tour-of-restate.zip
Setup your virtual environment:
python3 -m venv .venvsource .venv/bin/activate
Install the requirements:
pip install -r requirements.txt
Run the services:
python -m hypercorn --config hypercorn-config.toml tour/app/app:app
This GitHub repository contains the basic skeleton of the Python services that you develop in this tutorial.
- CLI
- wget
restate example rust-tour-of-restate && cd rust-tour-of-restate
wget https://github.com/restatedev/examples/releases/latest/download/rust-tour-of-restate.zip &&unzip rust-tour-of-restate.zip -d rust-tour-of-restate &&rm rust-tour-of-restate.zip && cd rust-tour-of-restate
Run the services
cargo run --bin app
Launch Restate
Restate is a single self-contained binary. No external dependencies needed. Check out our Local Dev page for instructions on how to install Restate Server, then do:
restate-server
You can find the Restate UI running on port 9070 (http://localhost:9070
) after starting the Restate Server.
Register the services with Restate
Now, we need to tell Restate where our services are running.
You can register services via the UI (http://localhost:9070
), the CLI or the Restate Admin API and supplying it the service endpoint URI:
- CLI
- curl
restate deployments register http://localhost:9080
curl localhost:9070/deployments --json '{"uri": "http://localhost:9080"}'
If you run Restate with Docker, replace http://localhost:9080
by http://host.docker.internal:9080
.
All set up!
- TypeScript
- Java
- Go
- Python
- Rust
In src/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
export const checkoutService = restate.service({name: "CheckoutService",handlers: {async handle(ctx: restate.Context, request: { userId: string; tickets: string[] }){return true;},}});export const CheckoutService: typeof checkoutService = { name: "CheckoutService"};
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The app.ts
file contains the definition of the endpoint that hosts the services.
In src/main/java/dev/restate/tour/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
@Servicepublic class CheckoutService {@Handlerpublic boolean handle(Context ctx, CheckoutRequest request) {return true;}}
Services are annotated by @Service
. A service consists of a set of handlers, and each handler is annotated by @Handler
.
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The AppMain.java
file contains the definition of the endpoint that hosts the services.
In app
you will find the skeletons of the various services to help you start implementing the app.
For example:
type CheckoutService struct{}type CheckoutRequest struct {UserId string `json:"userId"`Tickets []string `json:"tickets"`}func (CheckoutService) Handle(ctx restate.Context, request CheckoutRequest) (bool, error) {return true, nil}
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The main.go
file contains the definition of the endpoint that hosts the services.
In tour/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
@checkout.handler()async def handle(ctx: ObjectContext, order: Order) -> bool:return True
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The app.py
file contains the definition of the endpoint that hosts the services.
In src/app
you will find the skeletons of the various services to help you start implementing the app.
For example:
#[restate_sdk::service]pub(crate) trait CheckoutService {async fn handle(request: Json<CheckoutRequest>) -> Result<bool, HandlerError>;}pub struct CheckoutServiceImpl;impl CheckoutService for CheckoutServiceImpl {async fn handle(&self,_ctx: Context<'_>,Json(CheckoutRequest { user_id: _, tickets: _ }): Json<CheckoutRequest>,) -> Result<bool, HandlerError> {Ok(true)}}
You implement your Rust Restate service by implementing a trait which contains the handlers.
Restate handlers have the Restate Context supplied as the first argument. This is the entrypoint to the SDK.
The main.rs
file contains the definition of the endpoint that hosts the services.
Invoking Handlers
Handlers can be invoked in several ways: via HTTP requests, programmatically with the SDK, or via Kafka events.
Request-response calls over HTTP
Let's start with invoking our handler over HTTP.
We can either use the Restate UI playground or curl
.
- TypeScript
- Java
- Go
- Python
- Rust
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
.
In the UI, click on the CartObject
service, then on Playground
, and invoke your handler from there with body '"seat2B"'
.
Or via curl
:
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
.
Via curl
:
curl -X POST localhost:8080/CartObject/Mary/checkout
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
In the UI, click on the CartObject
service, then on Playground
, and invoke your handler from there with body '"seat2B"'
.
Or via curl
:
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
Via curl
:
curl -X POST localhost:8080/CartObject/Mary/checkout
For example, add a ticket seat2B
to the cart of Mary by calling the AddTicket
handler of the CartObject
:
In the UI, click on the CartObject
service, then on Playground
, and invoke your handler from there with body '"seat2B"'
.
Or via curl
:
curl localhost:8080/CartObject/Mary/AddTicket --json '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the Checkout
handler of the CartObject
:
Via curl
:
curl -X POST localhost:8080/CartObject/Mary/Checkout
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
In the UI, click on the CartObject
service, then on Playground
, and invoke your handler from there with body '"seat2B"'
.
Or via curl
:
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
Via curl
:
curl -X POST localhost:8080/CartObject/Mary/checkout
For example, add a ticket seat2B
to the cart of Mary by calling the addTicket
handler of the CartObject
:
In the UI, click on the CartObject
service, then on Playground
, and invoke your handler from there with body '"seat2B"'
.
Or via curl
:
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'
If this prints out true
, then you have a working setup.
When Mary wants to proceed with the purchase, call the checkout
handler of the CartObject
:
Via curl
:
curl -X POST localhost:8080/CartObject/Mary/checkout
We will use these two curl
commands often when developing the code, so keep them handy.
Restate acts as a proxy for your services. It forwards the request to the correct service and handler. Therefore, the request is sent to Restate and not directly to the service.
Mary
?Handlers are either a part of plain services or Virtual Objects.
Virtual Objects are a special type of service that allows you to group handlers together, share state between them, and control concurrency.
Each Virtual Object has a unique key.
We will cover the difference in more detail later.
For now, it's important to note that when invoking a handler within a Virtual Object, you need to specify its key.
In our example, the CartObject
and TicketObject
are Virtual Objects, while the CheckoutService
is a plain service.
To add the ticket to Mary's cart, we need to specify the key Mary
in the path to reach her Virtual Object.
We can do the same programmatically within a handler by using the SDK. Let's try this out!
Request-response calls between handlers
You can also call other handlers programmatically by using the clients generated by the Restate SDK. Let's try this out!
- TypeScript
- Java
- Go
- Python
- Rust
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
async addTicket(ctx: restate.ObjectContext, ticketId: string) {const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();return true;},
Service logs
- Create the client via
ctx.serviceClient
orctx.objectClient
(for Virtual Objects). Specify the service definition (TicketObject
) and optionally the Virtual Object key (ticketId
). - Specify the handler you want to call and supply the request. Here
reserve()
. - Await the response of the call.
Send a request to CartObject/addTicket
as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
@Handlerpublic boolean addTicket(ObjectContext ctx, String ticketId) {boolean reservationSuccess = TicketObjectClient.fromContext(ctx, ticketId).reserve().await();return true;}
Service logs
- Use the pre-generated client (
TicketObject
): This gets generated when you compile the code for the first time. So if you haven't done that yet, run./gradlew build
to generate the client. - Supply the context (and specify the Virtual Object) key via
fromContext(ctx, ticketId)
. - Specify the handler you want to call and supply the request. Here
reserve()
. - Await the response of the call.
Once you have added this to the code, restart the service, call the CartObject/addTicket
method as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/AddTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/Reserve
handler:
func (CartObject) AddTicket(ctx restate.ObjectContext, ticketId string) (bool, error) {reservationSuccess, err := restate.Object[bool](ctx, "TicketObject", ticketId, "Reserve").Request(restate.Void{})if err != nil {return false, err}return reservationSuccess, nil}
Service logs
- Create the client via
restate.Service
orrestate.Object
(for Virtual Objects). Specify the return type (bool
) as a type parameter, the service name (TicketObject
) and method (Reserve
) and where necessary the Virtual Object key (ticketId
). - Specify what type of call you want to make - in this case we want a synchronous request-response call so we use
Request()
, passingrestate.Void{}
as this handler needs no input type.
Send a request to CartObject/AddTicket
as we did previously, and have a look at the service logs.
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
@cart.handler("addTicket")async def add_ticket(ctx: ObjectContext, ticket_id: str) -> bool:reserved = await ctx.object_call(reserve, key=ticket_id, arg=None)return reserved
- Use
ctx.service_call
(for Services) orctx.object_call
(for Virtual Objects). - Specify the handler you want to call and supply the request: Here, we supply the
reserve
method that we import from theTicketObject
. For Virtual Objects, you also need to specify the key of the Virtual Object that you want to call. Here, this is the ticket ID. - Await the response of the call.
Send a request to CartObject/addTicket
as we did previously.
You can see the calls to addTicket
and reserve
in the Restate Server logs.
When we add a ticket to the cart, the CartObject/addTicket
handler first needs to reserve the ticket for the user.
It does that by calling the TicketObject/reserve
handler:
async fn add_ticket(&self,ctx: ObjectContext<'_>,ticket_id: String,) -> Result<bool, HandlerError> {let _reservation_success = ctx.object_client::<TicketObjectClient>(ticket_id.clone()).reserve().call().await?;Ok(true)}
- Create the client via
ctx.service_client
orctx.object_client
(for Virtual Objects). Specify the service client definition (TicketObjectClient
) and optionally the Virtual Object key (ticket_id
). - Specify the handler you want to call and supply the request. Here
reserve()
. - Await the response of the call.
Send a request to CartObject/addTicket
as we did previously, and have a look at the service logs.
Sending messages between handlers
We can also let handlers send messages to other handlers without waiting for a response.
- TypeScript
- Java
- Go
- Python
- Rust
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
async expireTicket(ctx: restate.ObjectContext, ticketId: string) {ctx.objectSendClient(TicketObject, ticketId).unreserve();},
Specify that you want to call the TicketObject
by supplying it to the objectSendClient
function.
Then call the unreserve
handler on the TicketObject
.
Once you have added this to the code, call the CartObject/expireTicket
handler:
curl localhost:8080/CartObject/Mary/expireTicket --json '"seat2B"'
Service logs
The service logs show how the expireTicket
handler gets executed and then the unreserve
handler.
The call to expireTicket
finishes earlier than the unreserve
handler because expireTicket
didn't wait for the response of the unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
@Handlerpublic void expireTicket(ObjectContext ctx, String ticketId) {TicketObjectClient.fromContext(ctx, ticketId).send().unreserve();}
You now call send()
on the generated client to send a message instead of doing a request-response call.
You also don't need to await the response, as you don't expect one.
Call the handler via:
curl localhost:8080/CartObject/Mary/expireTicket --json '"seat2B"'
Service logs
The service logs show how the expireTicket
handler gets executed and then the unreserve
handler.
The call to expireTicket
finishes earlier than the unreserve
handler because expireTicket
didn't wait for the response of the unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/ExpireTicket
handler is triggered.
Let the ExpireTicket
handler call the TicketObject/Unreserve
handler.
func (CartObject) ExpireTicket(ctx restate.ObjectContext, ticketId string) error {restate.ObjectSend(ctx, "TicketObject", ticketId, "Unreserve").Send(restate.Void{})return nil}
Specify that you want to call the TicketObject
by supplying the service and method names to the ObjectSend()
function.
This function is an alternative to Object()
which provides a client that can only send one-way messages
and therefore doesn't need an output type parameter. Finally call the Send
method on the returned client.
Once you have added this to the code, call the CartObject/ExpireTicket
handler:
curl localhost:8080/CartObject/Mary/ExpireTicket --json '"seat2B"'
Service logs
The service logs show how the ExpireTicket
handler gets executed and then the Unreserve
handler.
The call to ExpireTicket
finishes earlier than the Unreserve
handler because ExpireTicket
didn't wait for the response of the Unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
@cart.handler("expireTicket")async def expire_ticket(ctx: ObjectContext, ticket_id: str):ctx.object_send(unreserve, key=ticket_id, arg=None)
Import the unreserve
handler from the TicketObject
and supply it to ctx.object_send
together with the ticket ID.
Once you have added this to the code, call the CartObject/expireTicket
handler:
curl localhost:8080/CartObject/Mary/expireTicket --json '"seat2B"'
The Restate Server logs show how the expireTicket
handler gets executed and then the unreserve
handler.
In the example, when a seat gets added to the shopping cart, it gets reserved for 15 minutes.
When a user didn't proceed with the payment before the timeout, the CartObject/expireTicket
handler is triggered.
Let the expireTicket
handler call the TicketObject/unreserve
handler.
async fn expire_ticket(&self,ctx: ObjectContext<'_>,ticket_id: String,) -> Result<(), HandlerError> {ctx.object_client::<TicketObjectClient>(ticket_id).unreserve().send();Ok(())}
Specify that you want to call the TicketObject
by supplying it to the objectSendClient
function.
Then call the unreserve
handler on the TicketObject
.
Once you have added this to the code, call the CartObject/expireTicket
handler:
curl localhost:8080/CartObject/Mary/expireTicket --json '"seat2B"'
Restate Server logs
The Restate Server logs show how the expireTicket
handler gets executed and then the unreserve
handler.
The call to expireTicket
finishes earlier than the unreserve
handler because expireTicket
didn't wait for the response of the unreserve
handler.
Restate persists and retries failed one-way invocations. There is no need to set up message queues to ensure delivery!
To send messages via the UI or curl
, add /send
to the handler path:
- TypeScript
- Java
- Go
- Python
curl localhost:8080/CartObject/Mary/addTicket/send --json '"seat2B"'
curl localhost:8080/CartObject/Mary/addTicket/send --json '"seat2B"'
curl localhost:8080/CartObject/Mary/AddTicket/send --json '"seat2B"'
curl localhost:8080/CartObject/Mary/addTicket/send --json '"seat2B"'
This returns the invocation ID. This is a unique identifier for the invocation. You can use it to track the progress of the invocation via the CLI, and to correlate logs and metrics.
📝 Try it out
- TypeScript
- Java
- Go
- Python
- Rust
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Make the CartObject/Checkout
handler call the CheckoutService/Handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Make the CartObject/checkout
handler call the CheckoutService/handle
handler.
For the request field, you can use a hard-coded string array for now: ["seat2B"]
.
You will fix this later on. Note that the CheckoutService
is not a Virtual Object, so you don't need to specify a key.
Solution
Durable Execution
The calls we just did seem like regular RPC calls as you might know them from other service frameworks. But under the hood a lot more is happening.
Restate makes RPC calls resilient by letting the Restate Server and SDK cooperate. Restate tracks the execution of code in a journal and can replay it in case of a failure. This is called Durable Execution.
Have a look at the animation to understand what happened under-the-hood:
Journals
addTicket ( Mary, seat2B )
reserve ( seat2B )
This animation shows you what happened under the hood when we did the reserve call from the `CartObject` to the `TicketObject`. The animation uses the TypeScript SDK.
async function addTicket(ctx, ticketId){
const success = await ctx
.objectClient(ticketObject)
.reserve(ticketId);
return success;
}
Journal:
{ seat2B }
{ success }
{ success }
async function reserve(ctx, ticketId){
...
return success;
}
Journal:
{ success }
Whenever a failure would happen, Restate would be able to recover the latest state of the handler by sending over the journal. The code would fast-forward to the point where it crashed, and continue executing from there on.
- TypeScript
- Java
- Go
- Python
- Rust
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
async addTicket(ctx: restate.ObjectContext, ticketId: string) {const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();return true;},
Add the following code after the reservation call, to let the code throw an error after the call:
throw new Error("Failing");
Call CartObject/addTicket
again and have a look at the service logs.
Service logs
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
@Handlerpublic boolean addTicket(ObjectContext ctx, String ticketId) {boolean reservationSuccess = TicketObjectClient.fromContext(ctx, ticketId).reserve().await();return true;}
Instead of returning true, let the code fail after the call:
throw new IllegalStateException("The handler failed");
Call CartObject/addTicket
again and have a look at the service logs.
Service logs
To see the recovery of partial progress in practice, let's make the CartObject/AddTicket
handler crash right after the call.
func (CartObject) AddTicket(ctx restate.ObjectContext, ticketId string) (bool, error) {reservationSuccess, err := restate.Object[bool](ctx, "TicketObject", ticketId, "Reserve").Request(restate.Void{})if err != nil {return false, err}return reservationSuccess, nil}
Add the following code after the reservation call, to let the code throw an error after the call:
return false, fmt.Errorf("Failing")
Call CartObject/AddTicket
again and have a look at the service logs.
Service logs
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
@cart.handler("addTicket")async def add_ticket(ctx: ObjectContext, ticket_id: str) -> bool:reserved = await ctx.object_call(reserve, key=ticket_id, arg=None)return reserved
Add the following code after the reservation call, to let the code throw an error after the call:
raise Exception("Failing")
Call CartObject/addTicket
again and have a look at the Restate Server logs to see what happens.
To see the recovery of partial progress in practice, let's make the CartObject/addTicket
handler crash right after the call.
async fn add_ticket(&self,ctx: ObjectContext<'_>,ticket_id: String,) -> Result<bool, HandlerError> {let _reservation_success = ctx.object_client::<TicketObjectClient>(ticket_id.clone()).reserve().call().await?;Ok(true)}
Add the following code instead of Ok(true)
, to let the code throw an error after the call:
Err(HandlerError::from("Failing"))
Call CartObject/addTicket
again and have a look at the Restate Server logs.
Restate Server logs
You see the retries taking place. And you see that only the first time the call to the TicketObject
was made.
The other times, the call was skipped and the journaled response was replayed.
By default, Restate will keep retrying failed invocations until they succeed. If you want to cancel an invocation in a retry loop, you can use the CLI to do this. Let's have a look at that next.
Discover more about Restate error handling, retry strategies, and common patterns in the Error Handling guide.
Debugging with the UI and CLI
Now that we have a failing invocation, let's take the opportunity to show you how you can get more information about what is going on via the UI or CLI. Find the failing invocation in the list and have a look at it!
Debugging with the UI
The UI lets you debug invocations from the invocations tab, where you can find information on retry counts, error messages, the journal, the deployment, and more.
Debugging with the CLI
The CLI is a management tool that lets you interact with the Restate Server. You can use it to boostrap a new project, but also to get information about the services and invocations.
Have a look at some useful commands and try them out yourself.
List the services:
restate services list
List the ongoing invocations:
restate invocations list
You find the invocation that is retrying in the list.
Use its invocation ID (inv_...
), to dig deeper and describe the invocation:
restate invocations describe inv_1fmRNvSNVxNp5PTqHI4HLJ17HpxzhB3MEV
You can cancel the invocation, or kill it with --kill
:
restate invocations cancel --kill inv_1fmRNvSNVxNp5PTqHI4HLJ17HpxzhB3MEV
Cancelling lets you stop the invocation from retrying, to start running any compensations. Killing ends the invocation without running any compensations.
Remove the throwing of the exception from your code before you continue.
🚩 Explore the intermediate solution in part1
, and run it with:
- TypeScript
- Java
- Go
- Python
- Rust
npm run part1
./gradlew -PmainClass=dev.restate.tour.part1.AppMain run
go run ./part1
python3 -m hypercorn -b localhost:9080 tour/part1/app:app
cargo run --bin part1
Scheduling Async Tasks
When a handler calls another handler, Restate registers the call and makes sure it happens. You can also ask Restate to execute the call at a later point in the future, by adding a delay parameter to the call. Restate then registers the call and triggers it after the delay has passed.
In the application, a ticket gets reserved for 15 minutes. If the user doesn't pay within that time interval, then it becomes available again to other users.
- TypeScript
- Java
- Go
- Python
- Rust
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
async addTicket(ctx: restate.ObjectContext, ticketId: string) {const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();if (reservationSuccess) {ctx.objectSendClient(CartObject, ctx.key, {delay: 15 * 60 * 1000}).expireTicket(ticketId);}return reservationSuccess;},
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Service logs
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
@Handlerpublic boolean addTicket(ObjectContext ctx, String ticketId) {boolean reservationSuccess = TicketObjectClient.fromContext(ctx, ticketId).reserve().await();if (reservationSuccess) {CartObjectClient.fromContext(ctx, ctx.key()).send().expireTicket(ticketId, Duration.ofMinutes(15));}return reservationSuccess;}
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Service logs
Let the CartObject/AddTicket
handler call the CartObject/ExpireTicket
handler with a delay of 15 minutes:
func (CartObject) AddTicket(ctx restate.ObjectContext, ticketId string) (bool, error) {reservationSuccess, err := restate.Object[bool](ctx, "TicketObject", ticketId, "Reserve").Request(restate.Void{})if err != nil {return false, err}if reservationSuccess {restate.ObjectSend(ctx, "CartObject", restate.Key(ctx), "ExpireTicket").Send(ticketId, restate.WithDelay(15*time.Minute))}return reservationSuccess, nil}
To test it out, put the delay to a lower value, for example 5 seconds, call the AddTicket
function, and see in the logs how the call to CartObject/ExpireTicket
is executed 5 seconds later.
Service logs
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
@cart.handler("addTicket")async def add_ticket(ctx: ObjectContext, ticket_id: str) -> bool:reserved = await ctx.object_call(reserve, key=ticket_id, arg=None)if reserved:ctx.object_send(expire_ticket, key=ctx.key(), arg=ticket_id, send_delay=timedelta(minutes=15))return reserved
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the Restate Server logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Let the CartObject/addTicket
handler call the CartObject/expireTicket
handler with a delay of 15 minutes:
async fn add_ticket(&self,ctx: ObjectContext<'_>,ticket_id: String,) -> Result<bool, HandlerError> {let reservation_success = ctx.object_client::<TicketObjectClient>(ticket_id.clone()).reserve().call().await?;if reservation_success {ctx.object_client::<CartObjectClient>(ctx.key()).expire_ticket(ticket_id.clone()).send_after(Duration::from_secs(15 * 60));}Ok(reservation_success)}
To test it out, put the delay to a lower value, for example 5 seconds, call the addTicket
function, and see in the logs how the call to CartObject/expireTicket
is executed 5 seconds later.
Restate Server logs
Don't forget to set the delay back to 15 minutes.
Durable timers are a powerful feature that can be used to implement workflows, schedule async tasks, or plan background jobs. Restate makes them resilient to failures and ensures that they get executed. No extra infrastructure needed!
Another timer-like feature of the SDK is suspendable sleep. Restate will make sure that the function gets resumed after the specified duration has passed. When running on function-as-a-service platforms, your function can suspend in the meantime, so you don't pay for the wait time.
- TypeScript
- Java
- Go
- Python
- Rust
await ctx.sleep(15 * 60 * 1000);
ctx.sleep(Duration.ofMinutes(15));
if err := restate.Sleep(ctx, 15*time.Minute); err != nil {return err}
await ctx.sleep(delta=timedelta(minutes=15))
ctx.sleep(Duration::from_millis(15 * 60 * 1000)).await?;
🚩 Explore the intermediate solution in part2
, and run it with:
- TypeScript
- Java
- Go
- Python
- Rust
npm run part2
./gradlew -PmainClass=dev.restate.tour.part2.AppMain run
go run ./part2
python3 -m hypercorn -b localhost:9080 tour/part2/app:app
cargo run --bin part2
Virtual Objects vs. Services
At the beginning of this tutorial, we mentioned that the TicketObject
and CartObject
services are Virtual Objects.
Virtual Objects are identified by a key and allow you to store K/V state in Restate. For each Virtual Object (key), only one invocation can run at a time (across all the handlers of that Virtual Object).
Services, on the other hand, do not have access to K/V state, and handlers can run concurrently.
With access to consistent K/V state and strong concurrency guarantees, implementing the TicketObject
in a resilient and consistent way becomes straightforward.
When a user reserves a ticket, we want to be sure that no concurrent other requests are reserving the same ticket at the same time.
To get this behaviour, we key the TicketObject
on ticket ID. We now have a single Virtual Object per ticket.
If you do long-running operations in a Virtual Object, no other invocations are processed the meantime.
For example, if you would implement the expiration of the ticket in the CartObject
service by sleeping for 15 minutes:
- TypeScript
- Java
- Go
- Python
- Rust
await ctx.sleep(15 * 60 * 1000);ctx.objectSendClient(TicketObject, ticketId).unreserve();
ctx.sleep(Duration.ofMinutes(15));TicketObjectClient.fromContext(ctx, ticketId).send().unreserve();
if err := restate.Sleep(ctx, 15*time.Minute); err != nil {return err}restate.ObjectSend(ctx, "TicketObject", ticketId, "unreserve").Send(restate.Void{})
await ctx.sleep(delta=timedelta(minutes=15))ctx.object_send(unreserve, key=ticket_id, arg=None)
ctx.sleep(Duration::from_millis(15 * 60 * 1000)).await?;ctx.object_client::<TicketObjectClient>(ctx.key()).unreserve().send();
The user wouldn't be able to add any other tickets, nor buy the tickets. If you do a delayed call, the invocation isn't ongoing until the delay has passed, so the Virtual Object is not locked.
Consistent K/V state
Restate offers a key-value store to store application state for Virtual Objects.
Restate's state is guaranteed to be consistent across retries and invocations. This eliminates the need for a session database.
Getting and setting K/V state
- TypeScript
- Java
- Go
- Python
- Rust
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
async addTicket(ctx: restate.ObjectContext, ticketId: string) {const reservationSuccess = await ctx.objectClient(TicketObject, ticketId).reserve();if (reservationSuccess) {const tickets = (await ctx.get<string[]>("tickets")) ?? [];tickets.push(ticketId);ctx.set("tickets", tickets);ctx.objectSendClient(CartObject, ctx.key, {delay: 15 * 60 * 1000}).expireTicket(ticketId);}return reservationSuccess;},
To retrieve the cart, you use ctx.get
.
This returns null
if the value has never been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
// At the top of the class, define the state key: supply a name and (de)serializerpublic static final StateKey<Set<String>> STATE_KEY = StateKey.of("tickets", new TypeRef<>() {});@Handlerpublic boolean addTicket(ObjectContext ctx, String ticketId) {boolean reservationSuccess = TicketObjectClient.fromContext(ctx, ticketId).reserve().await();if (reservationSuccess) {Set<String> tickets = ctx.get(STATE_KEY).orElseGet(HashSet::new);tickets.add(ticketId);ctx.set(STATE_KEY, tickets);CartObjectClient.fromContext(ctx, ctx.key()).send().expireTicket(ticketId, Duration.ofMinutes(15));}return reservationSuccess;}
To retrieve the cart, you use ctx.get
with a state key that describes the name and the (de)serializers to be used.
ctx.get
returns an Optional, only containing a value if one was set before.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
Adapt the CartObject/AddTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
func (CartObject) AddTicket(ctx restate.ObjectContext, ticketId string) (bool, error) {reservationSuccess, err := restate.Object[bool](ctx, "TicketObject", ticketId, "Reserve").Request(restate.Void{})if err != nil {return false, err}if reservationSuccess {tickets, err := restate.Get[[]string](ctx, "tickets")if err != nil {return false, err}tickets = append(tickets, ticketId)restate.Set(ctx, "tickets", tickets)restate.ObjectSend(ctx, "CartObject", restate.Key(ctx), "ExpireTicket").Send(ticketId, restate.WithDelay(15*time.Minute))}return reservationSuccess, nil}
To retrieve the cart, you use restate.Get
.
This returns the zero value if there's no value for this key; a nil slice is a useful result in this case.
After you added the ticket to the cart array, you set the state to the new value with restate.Set
.
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
@cart.handler("addTicket")async def add_ticket(ctx: ObjectContext, ticket_id: str) -> bool:reserved = await ctx.object_call(reserve, key=ticket_id, arg=None)if reserved:tickets = await ctx.get("tickets") or []tickets.append(ticket_id)ctx.set("tickets", tickets)ctx.object_send(expire_ticket, key=ctx.key(), arg=ticket_id, send_delay=timedelta(minutes=15))return reserved
To retrieve the cart, you use ctx.get
.
This returns null
if the value has never been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
Adapt the CartObject/addTicket
function to keep track of the cart items.
After reserving the product, you add the ticket to the shopping cart.
Have a look at the highlighted code:
async fn add_ticket(&self,ctx: ObjectContext<'_>,ticket_id: String,) -> Result<bool, HandlerError> {let reservation_success = ctx.object_client::<TicketObjectClient>(ticket_id.clone()).reserve().call().await?;if reservation_success {let mut tickets = ctx.get::<Json<HashSet<String>>>("tickets").await?.unwrap_or_default().into_inner();tickets.insert(ticket_id.clone());ctx.set("tickets", Json(tickets));ctx.object_client::<CartObjectClient>(ctx.key()).expire_ticket(ticket_id.clone()).send_after(Duration::from_secs(15 * 60));}Ok(reservation_success)}
To retrieve the cart, you use ctx.get
.
This returns an Option
, containing the value if it has ever been set.
After you added the ticket to the cart array, you set the state to the new value with ctx.set
.
You can store multiple key-value pairs, by using different state keys.
Here, you get the value under the key "tickets"
.
Restate returns the cart belonging to the current Virtual Object (for example, user Mary
).
- TypeScript
- Java
- Go
- Python
- Rust
Run the services and call the addTicket
function, to see the interaction with state.
Service logs
Run the services and call the addTicket
function, to see the interaction with state.
Service logs
Run the services with TRACE loglevel (slog.SetLogLoggerLevel(-8)
in Go >= 1.22) and call the AddTicket
function, to see the interaction with state.
Service logs
When starting the invocation, Restate attaches the application state to the request. So when you operate on the state in your function, you get access to a local copy of the state for fast access.
- TypeScript
- Java
- Go
- Python
- Rust
Also adapt the CartObject/checkout
function, to use the tickets:
async checkout(ctx: restate.ObjectContext) {const tickets = (await ctx.get<string[]>("tickets")) ?? [];if (tickets.length === 0) {return false;}const success = await ctx.serviceClient(CheckoutService).handle({userId: ctx.key, tickets});if (success) {ctx.clear("tickets");}return success;},
After the tickets are checked out, you clear the state with ctx.clear
.
Also adapt the CartObject/checkout
function, to use the tickets:
@Handlerpublic boolean checkout(ObjectContext ctx) {Set<String> tickets = ctx.get(STATE_KEY).orElseGet(HashSet::new);if (tickets.isEmpty()) {return false;}boolean checkoutSuccess = CheckoutServiceClient.fromContext(ctx).handle(new CheckoutRequest(ctx.key(), tickets)).await();if (checkoutSuccess) {ctx.clear(STATE_KEY);}return checkoutSuccess;}
After the tickets are checked out, you clear the state with ctx.clear
.
Also adapt the CartObject/Checkout
function, to use the tickets:
func (CartObject) Checkout(ctx restate.ObjectContext) (bool, error) {tickets, err := restate.Get[[]string](ctx, "tickets")if err != nil || len(tickets) == 0 {return false, err}success, err := restate.Service[bool](ctx, "CheckoutService", "Handle").Request(CheckoutRequest{UserId: restate.Key(ctx), Tickets: []string{"seat2B"}})if err != nil {return false, err}if success {restate.Clear(ctx, "tickets")}return success, nil}
After the tickets are checked out, you clear the state with restate.Clear
.
Also adapt the CartObject/checkout
function, to use the tickets:
@cart.handler()async def checkout(ctx: ObjectContext) -> bool:tickets = await ctx.get("tickets") or []if len(tickets) == 0:return Falsesuccess = await ctx.service_call(handle, arg={'user_id': ctx.key(),'tickets': tickets})if success:for ticket in tickets:ctx.object_send(mark_as_sold, key=ticket, arg=None)ctx.clear("tickets")return success
After the tickets are checked out, you clear the state with ctx.clear
.
Also adapt the CartObject/checkout
function, to use the tickets:
async fn checkout(&self, ctx: ObjectContext<'_>) -> Result<bool, HandlerError> {let tickets = ctx.get::<Json<HashSet<String>>>("tickets").await?.unwrap_or_default().into_inner();if tickets.is_empty() {return Ok(false);}let success = ctx.service_client::<CheckoutServiceClient>().handle(Json(CheckoutRequest {user_id: ctx.key().parse()?,tickets: tickets.clone(),})).call().await?;if success {ctx.clear("tickets");}Ok(success)}
After the tickets are checked out, you clear the state with ctx.clear
.
Inspecting K/V state
Restate exposes information on invocations and application state.
You can watch the state of the CartObject
service, via:
- CLI
- psql
restate kv get -w -n 1 CartObject Mary
watch -n 1 'psql -h localhost -p 9071 -c "SELECT * FROM state WHERE service_name like '\''%CartObject%'\''";'
Add some tickets to the state to see how the query result gets updated.
Then, send a checkout request as earlier, and notice that the state is now empty.
📝 Try it out
- TypeScript
- Java
- Go
- Python
- Rust
Finishing CartObject/expireTicket
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Finishing CartObject/expireTicket
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Finishing CartObject/ExpireTicket
You have almost fully implemented the CartObject
. Let's finish CartObject/ExpireTicket
.
Before you call Unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/Unreserve
and remove it from the state.
Solution
Finishing CartObject/expireTicket
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Finishing CartObject/expireTicket
You have almost fully implemented the CartObject
. Let's finish CartObject/expireTicket
.
Before you call unreserve
, you first need to check if the ticket is still held by the user.
Retrieve the state and check if the ticket ID is in there.
If this is the case, then you call TicketObject/unreserve
and remove it from the state.
Solution
Implementing the TicketObject
Track the status of the tickets in the TicketObject
by storing it in the state and transitioning from one state to another, like a state machine.
The possible states are available (default), reserved, and sold.
Implement the handlers in the TicketObject
to reserve, unreserve, and mark a ticket as sold.
While you are developing them, monitor the state of the TicketObject
via:
- CLI
- psql
restate kv get -w -n 1 TicketObject seat2B
watch -n 1 'psql -h localhost -p 9071 -c "select * from state where service_name like '\''%TicketObject%'\''";'
- TypeScript
- Java
- Go
- Python
- Rust
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
TicketStatus.Available
, then change it toTicketStatus.Reserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatus.Available
, then returnfalse
.
Solution
TicketObject/unreserve
Clear the "status"
, if it's not equal to TicketStatus.Sold
.
Solution
TicketObject/markAsSold
Set the "status"
to TicketStatus.Sold
if it's reserved.
Solution
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
TicketStatus.Available
, then change it toTicketStatus.Reserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatus.Available
, then returnfalse
.
Solution
TicketObject/unreserve
Clear the "status"
, if it's not equal to TicketStatus.Sold
.
Solution
TicketObject/markAsSold
Set the "status"
to TicketStatus.Sold
if it's reserved.
Solution
TicketObject/Reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
auxiliary.TicketStatusAvailable
, then change it toTicketStatusReserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatusAvailable
, then returnfalse
.
Solution
TicketObject/Unreserve
Clear the "status"
, if it's not equal to auxiliary.TicketStatusSold
.
Solution
TicketObject/MarkAsSold
Set the "status"
to auxiliary.TicketStatusSold
if it's reserved.
Solution
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
"AVAILABLE"
, then change it to"RESERVED"
and returntrue
(reservation successful). - If the status isn't set to
"AVAILABLE"
, then returnfalse
.
Solution
TicketObject/unreserve
Clear the "status"
, if it's not equal to "SOLD"
.
Solution
TicketObject/markAsSold
Set the "status"
to "SOLD"
if it's reserved.
Solution
TicketObject/reserve
- Retrieve the value for the
"status"
state key. - If the value is set to
TicketStatus::Available
, then change it toTicketStatus::Reserved
and returntrue
(reservation successful). - If the status isn't set to
TicketStatus::Available
, then returnfalse
.
Solution
TicketObject/unreserve
Clear the "status"
, if it's not equal to TicketStatus::Sold
.
Solution
TicketObject/markAsSold
Set the "status"
to TicketStatus::Sold
if it's reserved.
Solution
🚩 Explore the intermediate solution in part3
, and run it with:
- TypeScript
- Java
- Go
- Python
- Rust
npm run part3
./gradlew -PmainClass=dev.restate.tour.part3.AppMain run
go run ./part3
python3 -m hypercorn -b localhost:9080 tour/part3/app:app
cargo run --bin part3
Journaling actions
Restate's Durable Execution mechanism tracks the progress of the code execution in a journal. Once an action/result has made it to the journal, it will not be re-executed on retries.
- TypeScript
- Java
- Go
- Python
- Rust
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
You can store the return value of any function in the journal, by using restate.Run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use restate.Run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/Handle
:
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
You can store the return value of any function in the journal, by using ctx.run
.
This lets you capture potentially non-deterministic computation and interaction with external systems in a safe way.
The SDK also offers helper functions for creating UUIDs and generating random numbers.
For the replay to work, code needs to be deterministic, otherwise the replayed entries do not line up with the code execution on retries.
So use ctx.run
to store the result of non-deterministic operations!
We can use this feature to do exactly-once payments in CheckoutService/handle
:
Generate an idempotency token
- TypeScript
- Java
- Go
- Python
- Rust
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
async handle(ctx: restate.Context,request: { userId: string; tickets: string[] }) {const idempotencyKey = ctx.rand.uuidv4();console.info("My idempotency key: " + idempotencyKey);throw new Error("Something happened!");return true;},
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
@Handlerpublic boolean handle(Context ctx, CheckoutRequest request) {String idempotencyKey = ctx.random().nextUUID().toString();System.out.println("My idempotency key: " + idempotencyKey);throw new IllegalStateException("The handler failed");}
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
func (CheckoutService) Handle(ctx restate.Context, request CheckoutRequest) (bool, error) {idempotencyKey := restate.Rand(ctx).UUID().String()ctx.Log().Info("Generated idempotency key", "idempotencyKey", idempotencyKey)return false, fmt.Errorf("Something happened!")}
Call CartObject/Checkout
and have a look at the logs to see what happens.
Service logs
Let's use ctx.run
to generate a unique payment identifier and store it in Restate.
Once the token is stored, it will be the same on retries.
Try it out by printing the idempotency key and then throwing an error:
@checkout.handler()async def handle(ctx: ObjectContext, order: Order) -> bool:idempotency_key = await ctx.run("idempotency_key", lambda: str(uuid.uuid4()))print("My idempotency key is: ", idempotency_key)raise Exception("Something happened!")return True
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
Let's use the SDK helper functions to generate a unique payment identifier and store it in Restate. Once the token is stored, it will be the same on retries. Try it out by printing the idempotency key and then throwing an error:
async fn handle(&self,mut ctx: Context<'_>,Json(request): Json<CheckoutRequest>,) -> Result<bool, HandlerError> {let idempotency_key = ctx.rand_uuid().to_string();info!("idempotent key: {}", idempotency_key);Err(HandlerError::from("Something happened!"))}
Call CartObject/checkout
and have a look at the logs to see what happens.
Service logs
Trigger the payment
- TypeScript
- Java
- Go
- Python
- Rust
Execute the payment via an external payment provider PaymentClient
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
async handle(ctx: restate.Context,request: { userId: string; tickets: string[] }) {const totalPrice = request.tickets.length * 40;const idempotencyKey = ctx.rand.uuidv4();const success = await ctx.run(() =>PaymentClient.get().call(idempotencyKey, totalPrice));return success;},
Execute the payment via an external payment provider via PaymentClient.get().call(idempotencyKey, amount)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
@Handlerpublic boolean handle(Context ctx, CheckoutRequest request) {double totalPrice = request.getTickets().size() * 40.0;String idempotencyKey = ctx.random().nextUUID().toString();boolean success =ctx.run(Boolean.class,() -> PaymentClient.get().call(idempotencyKey, totalPrice));return success;}
Execute the payment via an external payment provider via auxiliary.PaymentClient{}.Call(idempotencyKey, amount)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
func (CheckoutService) Handle(ctx restate.Context, request CheckoutRequest) (bool, error) {totalPrice := len(request.Tickets) * 40idempotencyKey := restate.Rand(ctx).UUID().String()if _, err := restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {return restate.Void{}, auxiliary.PaymentClient{}.Call(idempotencyKey, totalPrice)}); err != nil {return false, err}return true, nil}
Execute the payment via an external payment provider via payment_client.call(idempotency_key, total_price)
.
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollars.
@checkout.handler()async def handle(ctx: ObjectContext, order: Order) -> bool:total_price = len(order["tickets"]) * 40idempotency_key = await ctx.run("idempotency_key", lambda: str(uuid.uuid4()))async def pay():return await payment_client.call(idempotency_key, total_price)success = await ctx.run("payment", pay)return success
Execute the payment via an external payment provider PaymentClient
The payment provider will deduplicate payments based on the idempotency token.
We assume every ticket costs 40 dollarstypescript.
async fn handle(&self,mut ctx: Context<'_>,Json(CheckoutRequest { user_id, tickets }): Json<CheckoutRequest>,) -> Result<bool, HandlerError> {let total_price = tickets.len() as f64 * 40.0;let idempotency_key = ctx.rand_uuid().to_string();let pay_client = PaymentClient::new();let success = ctx.run(|| pay_client.call(&idempotency_key, total_price)).await?;Ok(success)}
📝 Try it out
Let's finish the checkout flow by sending the email notifications and marking the tickets as sold.
- TypeScript
- Java
- Go
- Python
- Rust
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient.get().notifyUserOfPaymentSuccess(request.getUserId())
. - Payment failure: notify the users via the
EmailClient.get().notifyUserOfPaymentFailure(request.getUserId())
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient.get().notifyUserOfPaymentSuccess(request.getUserId())
. - Payment failure: notify the users via the
EmailClient.get().notifyUserOfPaymentFailure(request.getUserId())
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/Handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient{}.NotifyUserOfPaymentSuccess(request.UserId)
. - Payment failure: notify the users via the
EmailClient{}.NotifyUserOfPaymentFailure(request.UserId)
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
email_client.notify_user_of_payment_success(order['user_id'])
. - Payment failure: notify the users via the
email_client.notify_user_of_payment_failure(order['user_id'])
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
Implement the email notifications
After the CheckoutService/handle
handler has handled the payment, you need to notify the users of the payment status:
- Payment success: notify the users via
EmailClient.notify_user_of_payment_success(&user_id)
. - Payment failure: notify the users via the
EmailClient.notify_user_of_payment_failure(&user_id)
.
Solution
Mark tickets as sold
Let the CartObject/checkout
handler mark all tickets as sold by calling TicketObject/markAsSold
for each ticket.
Solution
🥳 You have now fully implemented the ticket reservation system! Try it out by reserving some new tickets and buying them by checking out the cart.
🚩 Explore the intermediate solution in part4
, and run it with:
- TypeScript
- Java
- Go
- Python
- Rust
npm run part4
./gradlew -PmainClass=dev.restate.tour.part4.AppMain run
go run ./part4
python3 -m hypercorn -b localhost:9080 tour/part4/app:app
cargo run --bin part4
Idempotency for any request
As you saw, generating idempotency keys inside your handlers and storing them in Restate is easy. But this doesn't guard us yet against retries of the HTTP request to Restate.
- TypeScript
- Java
- Go
- Python
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the AddTicket
handler didn't receive the success response of its first request, it might retry the request.
For example, if the caller of the addTicket
handler didn't receive the success response of its first request, it might retry the request.
The second request will return false
because the ticket already got reserved the first time, but the caller won't know about this.
To cover this, you can add an idempotency-key
header to the incoming request to let Restate deduplicate them.
- TypeScript
- Java
- Go
- Python
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \--json '"seat2C"'
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \--json '"seat2C"'
In our example, when we call the CartObject/AddTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/AddTicket \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \--json '"seat2C"'
In our example, when we call the CartObject/addTicket
handler, the first time the response is true
and the second time it's false
.
However, if we use the same idempotency key, the second call will return true
as well, because it will return the result of the first call:
curl localhost:8080/CartObject/Mary/addTicket \-H 'idempotency-key: ad5472esg4dsg525dssdfa5loi' \--json '"seat2C"'
You can also see from the service logs that the handler wasn't executed the second time.
Restate gives you idempotency for any service, handler and request for free. No extra setup.
You only need this when invoking handlers over HTTP. When a handler calls another handler, Restate automatically takes care of the idempotency.
Tracing
Restate exposes OpenTelemetry traces of your invocations.
Run Jaeger
docker run -d --name jaeger -p 4317:4317 -p 16686:16686 jaegertracing/jaeger:2.4.0
Relaunch Restate with tracing enabled
- binary
- Docker
restate-server --tracing-endpoint http://localhost:4317
docker run --name restate_dev -p 8080:8080 -p 9070:9070 -p 9071:9071 \-e RESTATE_TRACING_ENDPOINT=http://host.docker.internal:4317 \--add-host=host.docker.internal:host-gateway docker.restate.dev/restatedev/restate:1.3
Send a few requests
- TypeScript
- Java
- Go
- Python
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
curl localhost:8080/CartObject/Mary/AddTicket --json '"seat2A"'curl localhost:8080/CartObject/Mary/AddTicket --json '"seat2B"'curl localhost:8080/CartObject/Mary/AddTicket --json '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/Checkout
curl localhost:8080/CartObject/Mary/addTicket --json '"seat2A"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2B"'curl localhost:8080/CartObject/Mary/addTicket --json '"seat2C"'curl -X POST localhost:8080/CartObject/Mary/checkout
Go to the Jaeger UI
Inspect the traces
Select the CartObject
service from the service dropdown.
- TypeScript
- Java
- Go
- Python
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You should see the AddTicket
and Checkout
requests listed.
Have a look at the traces of the Checkout
call:
You should see the addTicket
and checkout
requests listed.
Have a look at the traces of the checkout
call:
You can see the calls that were done to Restate, for example invoke, sleep, one way call, get state, etc., and their timings. If you expand one of the traces, you can see tags describing some metadata of the context call, for example invocation ID and the request.
For more information, have a look at the tracing docs.
🏁 The end
You reached the end of this tutorial!
Let's recap what you did! You have built a ticket reservation system that is resilient, consistent, and scalable. We used Restate to provide us with durable, distributed building blocks to simplify the implementation of the system. Let's list a few of them:
What you implemented | What you didn't implement, as Restate handles it for you |
---|---|
✅ Request-response invocations | ❌ Handling retries, timeouts, etc. |
✅ Sending messages | ❌ Deploy and operate message queues for async requests |
✅ Idempotent HTTP calls | ❌ Write deduplication logic |
✅ Durable Execution: retries, partial progress recovery, and suspensions | ❌ Manual retry logic and partial progress recovery |
✅ Durable timers: sleeping and scheduling async tasks | ❌ Workflow orchestrators or cron jobs for scheduling tasks |
✅ Virtual Objects: concurrency guarantees and shared state | ❌ Guards for keeping state consistent across retries, concurrent requests, and scaling out. |
✅ K/V state: storing and inspecting | ❌ Session databases for state. State consistency guards. |
✅ Storing computation results in the journal | ❌ Logic to make operations idempotent (e.g. generate idempotency keys) |
You now know the essentials to start developing Restate services! Have a look at the next steps to explore further.
Next steps
- Run the examples
- Read the Concepts: although most of this has been covered in this tutorial
- Check out the SDK documentation