betim
betim12mo ago

Detecting batch queries/mutations on the server-side

Hello all, Looking to leverage batching as a means to streamline some of our cross-service requests, both query and mutation. We have a graph of interconnected tRPC services, some of which form trees of calls where service A calls service B, and service B then calls service C. Looking to get some guidance on how to identify on the receiving side whether a particular method invocation is a part of a batch request so that I could then batch requests to a downstream service, eg. I receive 10 queries in a batch to service A, which I would then like to batch in a single call to service B, and so on. Specifically: 1. Is there a mechanism I could introspect to tell that the current method invocation (query or mutation) is part of a batch request? 2. Is there a mechanism I could use to get access to the input and context of the other requests of the same type in the batch? Thanks!
7 Replies
Nick
Nick12mo ago
The best/only way might be to have a singleton hosting your own batching layer, then have the batched procedures call through that and let it batch them up But batching is an optimisation which sometimes lands, not something you should expect to behave deterministically. If you want to do this I would write a procedure which receives an array of inputs and then you send your batch from the client in one call
betim
betim12mo ago
thanks @Nick Lucas . I was hoping to avoid writing a batch-specific procedure, in addition to the single input use cases (both of which are relevant in different use-cases for us), but sounds like that isn't an option. I suppose if I'm gonna have to go down this route, one thing that would help is support for "overloading" in a TypeScript sense, in that I'd like to have a single named procedure (with two implementations) that supports a single input (producing a single output) as well as an array of inputs (producing an array of outputs). Is something like this supported in tRPC?
Nick
Nick12mo ago
Just use a different procedure name createItem createItems Overloads are nice, but JS/TS don't actually support them, it's always faked
betim
betim12mo ago
would be helpful from a DX standpoint for us but I'll go ahead with the separate procs for each case. thanks again!
Nick
Nick12mo ago
Well you could use a Zod discriminatedUnion for the input and output, but TS wouldn't know which output matches to which input
betim
betim12mo ago
yeah, that was one of the first approaches I was looking at but it foregoes the output type-safety I wanted ideally, something like this is the DX I was looking for https://www.typescriptlang.org/docs/handbook/functions.html#overloads for tRPC, using Zod-based schemas, instead of TS types, I'm sure something like it could be supported.. but I do realize it's a fairly niche case
Nick
Nick12mo ago
TS overloads only offer type safety for the consumer. The function itself has no idea what it’s been sent and has to check at runtime The only way to guarantee type safety on both sides is to use a different function name with its own impl