Unified Interface and Endpoints

• Starting a Server
• Connecting a Client
• Mode Checking
• EndpointMap: Token Routing
• WellKnownToken
• RequestStream
• ReplyPromise and ReplyFuture
• ReplyError
• Putting It Together

The #[service] macro generates the types. Now we need to understand how to wire them up: how the server registers with the transport, how the client connects, and how the endpoint routing system delivers messages to the right place.

Starting a Server

The generated struct provides two patterns. The simple path uses serve(), which spawns background tasks and returns a handle:

#![allow(unused)]
fn main() {
// Dynamic tokens (each instance gets unique random base):
let server = Calculator::init(&transport);

// Or well-known tokens (deterministic, no discovery needed):
let server = Calculator::well_known(&transport, WLTOKEN_CALC);

let handle = server.serve(Arc::new(CalculatorImpl), &providers);
// Tasks run until handle is dropped or stop() is called
}

init() allocates a random base token and registers all method endpoints with the transport’s EndpointMap. Each method gets its own InterfaceMethod in local mode, backed by a RequestStream with a NetNotifiedQueue that receives incoming request envelopes. The transport is bound at construction, so no &transport threading is needed in call sites.

serve() consumes the interface and spawns one task per method. Each task loops on recv(), dispatches to the handler, and sends the response back through the ReplyPromise. The returned ServerHandle holds close functions for each stream. Dropping it or calling stop() closes the streams, which causes the tasks to exit cleanly.

For more control, you can skip serve() and process each method manually:

#![allow(unused)]
fn main() {
let server = Calculator::init(&transport);
// Handle the `add` method yourself
while let Some((req, reply)) = server.add.recv().await {
    reply.send(AddResponse { result: req.a + req.b });
}
}

Connecting a Client

Clients are constructed from a base token (obtained via discovery or well-known addressing). The transport is bound at construction, so delivery methods require only the request payload:

#![allow(unused)]
fn main() {
// From a well-known token (both sides know the constant):
let calc = Calculator::client_well_known(server_address, WLTOKEN_CALC, &transport);

// From a discovered base token (received via serialization):
let calc = Calculator::from_base(server_address, base_token, &transport);
}

Each InterfaceMethod field in remote mode carries the destination address, method UID, and a reference to the transport. The codec is a transport-level concern, set at builder time. The delivery mode is explicit at the call site: get_reply for at-least-once, try_get_reply for at-most-once, send for fire-and-forget. See Delivery Modes for the full set.

#![allow(unused)]
fn main() {
let resp = calc.add.get_reply(AddRequest { a: 1, b: 2 }).await?;
assert_eq!(resp.result, 3);
}

Under the hood, get_reply creates a temporary ReplyFuture registered at a unique endpoint, then the response arrives as a packet routed to that endpoint.

Mode Checking

Every InterfaceMethod knows whether it’s in local or remote mode. Calling server methods (recv(), try_recv()) on a remote-mode field, or client methods (get_reply(), send()) on a local-mode field, will panic. Use is_remote() on the interface struct or on individual fields to check:

#![allow(unused)]
fn main() {
let calc = Calculator::init(&transport);
assert!(!calc.is_remote());  // server mode

let calc = Calculator::from_base(addr, token, &transport);
assert!(calc.is_remote());   // client mode
}

EndpointMap: Token Routing

The EndpointMap is the routing table at the heart of NetTransport. When a packet arrives with a token, the transport looks it up here to find the receiver.

It uses a hybrid lookup strategy:

• Well-known endpoints use O(1) array access. The first 64 token indices are reserved for system endpoints. WellKnownToken::Ping (index 1) is used for health monitoring, WellKnownToken::EndpointNotFound (index 0) handles unroutable messages.
• Dynamic endpoints use a BTreeMap<UID, Arc<dyn MessageReceiver>>. These are allocated at runtime for service methods and request-response correlation.

#![allow(unused)]
fn main() {
// Well-known: O(1) array lookup
map.insert_well_known(WellKnownToken::Ping, receiver)?;

// Dynamic: BTreeMap lookup (service methods, reply endpoints)
map.insert(base_token.adjusted(1), receiver);
}

Well-known endpoints cannot be removed. Dynamic endpoints can be registered and deregistered as services come and go.

WellKnownToken

The WellKnownToken enum defines system-level endpoints:

A well-known UID has first == u64::MAX and second equal to the token index. The is_well_known() method checks this, letting the endpoint map take the fast array path.

RequestStream

RequestStream<Req, Resp> is the server-side abstraction for receiving typed requests. In the unified interface, each local-mode InterfaceMethod wraps one. The stream contains a NetNotifiedQueue that the transport pushes incoming packets into, plus a bound reference to the transport. When you call recv(), it awaits the next RequestEnvelope<Req> from the queue and returns the deserialized request paired with a ReplyPromise.

The RequestEnvelope bundles the request payload with a reply_to endpoint, the address where the client is listening for the response:

#![allow(unused)]
fn main() {
struct RequestEnvelope<T> {
    request: T,
    reply_to: Endpoint,
}
}

ReplyPromise and ReplyFuture

These two types form the request-response correlation mechanism.

ReplyPromise<T> lives on the server side. When the server finishes processing a request, it calls reply.send(response) to serialize and deliver the response to the client’s reply_to endpoint. If the promise is dropped without being fulfilled, it automatically sends a ReplyError::BrokenPromise to the client so the client does not hang forever.

#![allow(unused)]
fn main() {
// Server side
let (req, reply) = stream.recv().await?;
reply.send(AddResponse { result: req.a + req.b });
}

ReplyFuture<T> lives on the client side. It implements Future and resolves when the server’s response arrives at the temporary endpoint that send_request registered. The future polls a NetNotifiedQueue for the response. If the queue is closed (connection failure), it resolves with the appropriate ReplyError.

ReplyFuture implements Drop to close its queue when the future is cancelled or goes out of scope. This prevents leaked wakers and ensures the temporary endpoint is cleaned up even if the caller abandons the request. Without this, a killed process would leave orphaned reply queues that hang forever.

Both types are Send + Sync + 'static. Internally they hold Arc<RwLock<...>> (or Arc<NetNotifiedQueue<...>> for the future’s reply channel), so you can move them across tokio::spawn boundaries, store them in Arc-shared state, and compose them with the broader Rust async ecosystem without contortions. The simulation runtime still runs on a single OS thread for determinism (new_current_thread().build()), but the Send bounds are a compile-time API contract, not a runtime cost. With only one task ever holding a given lock at a time on a single-thread runtime, there is no measurable contention.

ReplyError

The ReplyError enum covers every failure mode in the request-response lifecycle:

MaybeDelivered is the most important variant. It maps directly to FoundationDB’s request_maybe_delivered (error 1030). Instead of hiding delivery ambiguity behind a generic timeout, it tells you explicitly: the connection failed and we do not know whether the server processed your request. See Delivery Modes for how each delivery function produces this error and Designing Simulation-Friendly RPC for strategies to handle it.

Putting It Together

Here is the complete flow for a single RPC call:

1. Client calls calc.add.get_reply(req), which calls send_request
2. send_request creates a ReplyFuture at a unique temporary endpoint and registers it in the EndpointMap
3. The request is serialized as a RequestEnvelope with the temporary endpoint as reply_to, then sent to the server’s method endpoint (base.adjusted(1))
4. Transport routes the packet to the server’s RequestStream via the EndpointMap
5. Server receives (AddRequest, ReplyPromise) from the stream
6. Server calls reply.send(AddResponse { ... }), which serializes and sends to the reply_to endpoint
7. Transport routes the response packet to the client’s temporary endpoint
8. ReplyFuture resolves with the deserialized AddResponse
9. The temporary endpoint is deregistered from the EndpointMap

All of this happens over the same Peer connections and wire format we covered in previous chapters. In simulation, every step goes through the SimWorld event queue, making the entire RPC flow deterministic and subject to chaos injection.