Building a GenAI Service

The Reason

  • The Gemini API is undoubtedly the best free LLM API, with a large quota and few restrictions.

  • I used to think that it was not compatible with the OpenAI API.

  • Yesterday, I accidentally saw on the official website of the Gemini API that if you use the following base_url:

    β€œhttps://generativelanguage.googleapis.com/v1beta/openai/”

    You can use the OpenAI library (Python and TypeScript/JavaScript) and the REST API to access the Gemini model.

  • This makes the Gemini API much more playable 😊.

Building the Service

  • The first thing I thought of was to deploy it on Deno Deploy, so I can connect directly after reverse proxying 😍.
  • This time I didn’t use Claude 3.5 sonnet, but Gemini-1.5-pro, after all, it’s their own thing.
  • The funny thing is, at the beginning, Gemini flatly denied the existence of such a compatible API;
  • I directly threw the base_url to it, and then it started to work;
  • After writing the code, it kept reminding me that this compatible API may not exist, so the code may not work properly, and I should use Gemini’s own API instead. πŸ˜‚
  • Fortunately, it only ran into an error once before it worked normally. It is indeed one of the top three LLMs at present πŸ‘.

Source Code

  • The code was all written by Gemini and entered directly in the Deno Deploy playground;

  • Then go to settings to set two environment variables:

    • REAL_GEMINI_API_KEY is your real Gemini API key;
    • CUSTOM_AUTH_KEY is the verification code used to replace the real Gemini API key;

  • In this way, when using this reverse proxy API, you can hide your own real Gemini API key and add a layer of security protection.

  • The code is as follows:

const GEMINI_API_BASE_URL = "https://generativelanguage.googleapis.com/";
const CUSTOM_AUTH_KEY = Deno.env.get("CUSTOM_AUTH_KEY");
const REAL_GEMINI_API_KEY = Deno.env.get("REAL_GEMINI_API_KEY");
const OPENAI_COMPATIBLE_PREFIX = "/v1beta/openai";

Deno.serve(async (request) => {
  const url = new URL(request.url);

  // 1. Verify the user's API key
  const authHeader = request.headers.get("Authorization");
  if (!authHeader?.startsWith("Bearer ") || authHeader.substring(7) !== CUSTOM_AUTH_KEY) {
    return new Response("Unauthorized: Invalid API key", { status: 401 });
  }

  // 2. Build the new URL
  const newPathname = url.pathname.startsWith(OPENAI_COMPATIBLE_PREFIX)
    ? url.pathname.substring(OPENAI_COMPATIBLE_PREFIX.length)
    : url.pathname;
  const newURL = new URL(OPENAI_COMPATIBLE_PREFIX + newPathname, GEMINI_API_BASE_URL);

  // 3. Modify the request headers
  const headers = new Headers(request.headers);
  headers.set("Authorization", `Bearer ${REAL_GEMINI_API_KEY}`);

  // 4. Create a new request and forward it
  const newRequest = new Request(newURL.toString(), {
    method: request.method,
    headers: headers,
    body: request.body,
    redirect: "follow",
  });

  // 5. Forward the request and return the response
  const response = await fetch(newRequest);
  return new Response(response.body, {
    status: response.status,
    headers: response.headers,
  });
});