rustclan•16mo ago

TRPC ratelimiting endpoints

I am currently having some problems with a race condition in my TRPC nextJS api. Essentially what is happening is I have a enforceGuildPermissions method, which basically checks if the user who is making the request has permission to get the data for that guild. The data is stored in my Redis cache for 3 seconds. This works okay sometimes, but other times because there is 3-4 different trpc requests running for a single page which are guild, role and channel. It causes the last request (channel) to get rate limited by the discord API because they are all running concurrently, this means it doesn't give my caching code chance to update it before the next one runs. middleware route procedure:

const enforceGuildPermissions = enforceUserLoggedIn.unstable_pipe(
  async ({ ctx, next, rawInput }) => {
    const guildId: unknown = (rawInput as { guildId?: unknown })?.guildId;
    if (!guildId) throw new TRPCError({ code: 'BAD_REQUEST' });

    const webUser = await cache.webUsers.get(ctx.session.user.id);
    let guilds = webUser?.guilds;
    if (!guilds) {
      guilds = await getUserGuilds(ctx.session)

    }

    if (!guilds) throw new TRPCError({ code: 'UNAUTHORIZED' });
    
    // if the user is not in the guild return unauth
    const foundGuild = guilds.find((guild) => guild.id === guildId);
    if (!foundGuild) throw new TRPCError({ code: 'UNAUTHORIZED' });

    return next({
      ctx: {
        session: { ...ctx.session, user: ctx.session.user },
      },
    });
  }
);

export const guildProcedure = t.procedure.use(enforceGuildPermissions);

const enforceGuildPermissions = enforceUserLoggedIn.unstable_pipe(
  async ({ ctx, next, rawInput }) => {
    const guildId: unknown = (rawInput as { guildId?: unknown })?.guildId;
    if (!guildId) throw new TRPCError({ code: 'BAD_REQUEST' });

    const webUser = await cache.webUsers.get(ctx.session.user.id);
    let guilds = webUser?.guilds;
    if (!guilds) {
      guilds = await getUserGuilds(ctx.session)

    }

    if (!guilds) throw new TRPCError({ code: 'UNAUTHORIZED' });
    
    // if the user is not in the guild return unauth
    const foundGuild = guilds.find((guild) => guild.id === guildId);
    if (!foundGuild) throw new TRPCError({ code: 'UNAUTHORIZED' });

    return next({
      ctx: {
        session: { ...ctx.session, user: ctx.session.user },
      },
    });
  }
);

export const guildProcedure = t.procedure.use(enforceGuildPermissions);

12 Replies

rustclan•16mo ago

function which needs to be ratelimited (using the redis cache)

export const getUserGuilds = async (
  session: Session
): Promise<CachedUserGuild[] | null> => {
  if (!session.user.accessToken || !session.user.id) return null;

  const webUser = await cache.webUsers.get(session.user.id);
  if (webUser) return webUser.guilds;

  const response = await fetch(discord...)
  const guilds = await response.json();
  if (!response.ok || guilds.length <= 0) return null;
  
  // add guilds to cache
  await cache.webUsers.create(session.user.id, guilds);

  return guilds;
};

export const getUserGuilds = async (
  session: Session
): Promise<CachedUserGuild[] | null> => {
  if (!session.user.accessToken || !session.user.id) return null;

  const webUser = await cache.webUsers.get(session.user.id);
  if (webUser) return webUser.guilds;

  const response = await fetch(discord...)
  const guilds = await response.json();
  if (!response.ok || guilds.length <= 0) return null;
  
  // add guilds to cache
  await cache.webUsers.create(session.user.id, guilds);

  return guilds;
};

A big band aid fix would be to just add an artificial wait:

    if (!guilds) {
      await new Promise((resolve) => setTimeout(resolve, 1000));
      webUser = await cache.webUsers.get(ctx.session.user.id);
      guilds = webUser?.guilds;
      if (!guilds) {
        throw new TRPCError({ code: 'UNAUTHORIZED' });
      }
    }

    if (!guilds) {
      await new Promise((resolve) => setTimeout(resolve, 1000));
      webUser = await cache.webUsers.get(ctx.session.user.id);
      guilds = webUser?.guilds;
      if (!guilds) {
        throw new TRPCError({ code: 'UNAUTHORIZED' });
      }
    }

But obviously this is not very elegant.. bump

Nick•16mo ago

This isn’t really a tRPC problem, caching and rate limiting is hard Probably simplest to recognise rate limits and retry the task though

Alex / KATT 🐱•16mo ago

You can dedupe the call to discord Have a fn to do it passed down through the context and memoize the promise

Alex / KATT 🐱•16mo ago

https://www.typescriptlang.org/play?#code/FDBmFcDsGMBcEsD2kAEBbApmxAxSAeAFRQwA9YNIATAZxQAoA6ZgQwCcBzGgLhRcgCeAbQC6AShQBeAHx9B0+ojbwOvQmLUoA3sBQpoyGrHRZEvAEoYDbKviPLIHADRyBsydoC+IPQchGUUFQPJlZOHldRCRltXT19Q2MAawwBKRQAKQBlAHkAOUZ7eEd4UAF6di4xAG44vVKGAEJ6FLTik2wxCR14+MxsIVaRdKUVUMZKmhq6lG9en3i2DFhwNlR+xEHUkVq9bzillbXA1BY6Qlr94D8AqngaayosgBEAaXSelCpELMRMWAAFsUOPRigAHcCwXhFRzdGY3RAAGwwjERiBBAHJoCxEciqCgAO7wQEYlzgyHTPbAfY3YzfX7-IGOdIbPD0O4PJRPN6Mel-ZZMjjTYB8xnA+gY0CIRAYsQin78wHiyXS2XyhkC5VSmVy9WKwUSgBG7DVwCAA

TS Playground - An online editor for exploring TypeScript and JavaS...

The Playground lets you write TypeScript or JavaScript online in a safe and sharable way.

Alex / KATT 🐱•16mo ago

try running that

rustclan•16mo ago

Hi, thank you for the response. But I'm not sure I follow. Since this is hosted on serverless infrastructure (vercel), I don't think this is possible to use a in memory cache to stop this from happening? I may be misunderstanding your solution though.

Alex / KATT 🐱•16mo ago

You can use the above to make sure two calls to discord aren't done within the same request

Alex / KATT 🐱•15mo ago

look at line27 here https://www.typescriptlang.org/play?#code/JYWwDg9gTgLgBAKjgQwM5xlMBjAcgUwA94AzKCEOAcgAFMcB6VfKANxYeQBNkwYXUDAHZEYVAFDiSAVyHYYwCELgh8ICADEhAHgAqcUfiFd0ACgB0l5FADmqAFwohATwDaAXQCUcALwA+J2c-U2hgG0ddTwi4AG9xODhsJVR4VXVHACV8JKgubRSoYCEbABpAgJ9YgF9JBKShFLgSZUqLK1sHQI9vf1j4hMTk+ABrfGdfOAApAGUAeVxzAqKbYBJnU2s7TwBufoTVuFMAQlNR8aKVNQhPbziBgbSIVzP3CdCbNvNN1B29uBr7rUBlB8DBpFBlI9nmN3LsEjV+iCwRCmso0HBdLsEeJ6o0uMBUDkuNMACIAaQmdzgXAg0wooIAFstTEUwNIYI4lsVbn9cRAADb4cz8iAfKjYZD8wVcOAAd2AMAZVDKrPZv3h4gRDAYcAAogANACCAFkAAoAGV19iksnkimU2BByH4AGElPxiCE+J16HhROYXU7+ARiG6hB6YLM+Pafn0EkjwcoqQkaXTVIrlo5LuotKZ8YToMTyeZU-SM9ySv0ajUgA

TS Playground - An online editor for exploring TypeScript and JavaS...

The Playground lets you write TypeScript or JavaScript online in a safe and sharable way.

Alex / KATT 🐱•15mo ago

if you use that the thing will only be called once per request so batching will be deduped if you have many users you gotta do something fancier where you can synthesise the promise in redis or something

rustclan•15mo ago

Thank you very much, when I get some time I shall look at this. @alex / KATT hi, I had time to do this now. Thank you SO much! This issue was slowing my site down alot, and now it is fast once again! But just so I understand, why does this solution not scale? You say I may have to deal with the promise in redis? Why is using memo not okay?

Nick•15mo ago

I haven’t looked at the solution, but (as a thought experiment) what happens if you have 2, 3, 15 parallel instances of the API running? (Horizontal scaling)

rustclan•15mo ago

Hmm, yes okay. I see what you are saying. This depends on vercel runs nextjs api endpoints, im not entirely sure. For now, this is working just fine. But I should look into this just encase. Thank you very much for the help 🙂