Invoke Vertex AI Agent in Firebase App

Unlock the Power of AI in Firebase App with Vertex AI Agent

I create Vertext AI agent and deploy to Agent Engine. I usually would create Streamlit or Gradio Python application to invoke it, usually it involves finding correct code sample in Python. However, my client wants to use Firebase and application hosting, which uses Typescript. Interestingly, most code samples are just calling Gemini models from Firebase, so I document here the current finding on how to call custom Vertex AI Agent.

Firebase

Firebase recently supports vibe-coding and supports AI logic. For common tasks, people can provide requirements prompt and it generates the code.

AI-powered browser-based development

Currently, you need to use REST way to interact with Vertex AI Agent as not like in Python, there is “remote_agent = agent_engines.get; response = remote_agent.query(input=customer_question)”, no such thing in Typescript.

So first make sure below works.

REST agent query

curl \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/reasoningEngines/RESOURCE_ID:query -d '{
  "class_method": "query",
  "input": {
    "input": "What is the exchange rate from US dollars to Swedish Krona today?"
  }
}'

Typescript: used rest api to talk about agent without using library.

import { GoogleAuth } from 'google-auth-library';
import * as fs from 'fs';
import * as path from 'path';

       const fileDataUri = input.evidenceDataUris[0];
       
       let base64Data = fileDataUri.substring(fileDataUri.indexOf(',') + 1);
       const fileContent = Buffer.from(base64Data, 'base64').toString('utf8');

       const keyFilePath = path.join(process.cwd(), 'src', 'ai', 'flows', '<service account key file>');
       const keyFileContent = fs.readFileSync(keyFilePath, 'utf8');
       const credentials = JSON.parse(keyFileContent);

       const auth = new GoogleAuth({
          credentials,
          scopes: 'https://www.googleapis.com/auth/cloud-platform',
       });
       
       const authToken = await auth.getAccessToken();

       const agentEndpoint = 'https://us-central1-aiplatform.googleapis.com/v1/projects/<project-id>/locations/us-central1/reasoningEngines/<endpoint>:query';

       const response = await fetch(agentEndpoint, {
          method: 'POST',
          headers: {
            'Authorization': `Bearer ${authToken}`,
            'Content-Type': 'application/json',
          },
          body: JSON.stringify({
            class_method: 'query',
            input: {
              input: fileContent,
            },
          }),
       });

       if (!response.ok) {
          const errorBody = await response.text();
          throw new Error(`Agent request failed with status ${response.status}: ${errorBody}`);
       }
       
       const responseData = await response.json();
       let resultText = "Could not parse agent response.";
        try {
            // The agent's raw output is a JSON string within the 'output' key of the response.
            if (responseData && responseData.output) {
                // First, parse the outer JSON which contains the 'output' key.
                const agentOutputString = responseData.output;
                // The content of 'agentOutputString' is itself a JSON string. Parse it.
                const agentOutputJson = JSON.parse(agentOutputString);
                // The actual markdown is in the 'output' key of this inner JSON.
                resultText = agentOutputJson.output || "Agent returned a response, but the 'output' key was empty.";
            }
        } catch (e) {
            console.error("Failed to parse agent JSON response:", e);
            // Fallback to the raw string if parsing fails.
            resultText = responseData?.output || JSON.stringify(responseData);
        }

       return {
         type: 'agent',
         testResult: resultText,
       };

Failed tries

firebase genkit

Can call gemini model, but does not provide way to call Agent engine agent

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/googleai';

const ai = genkit({ plugins: [googleAI()] });

const { text } = await ai.generate({
    model: googleAI.model('gemini-2.0-flash'),
    prompt: 'Why is Firebase awesome?'
});

nodejs-vertexai

Can call gemini model, but does not provide way to call Agent engine agent

const {
  FunctionDeclarationSchemaType,
  HarmBlockThreshold,
  HarmCategory,
  VertexAI
} = require('@google-cloud/vertexai');

const project = 'your-cloud-project';
const location = 'us-central1';
const textModel =  'gemini-1.5-flash';
const visionModel = 'gemini-1.5-flash';

const vertexAI = new VertexAI({project: project, location: location});

// Instantiate Gemini models
const generativeModel = vertexAI.getGenerativeModel({
    model: textModel,
    // The following parameters are optional
    // They can also be passed to individual content generation requests
    safetySettings: [{category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE}],
    generationConfig: {maxOutputTokens: 256},
    systemInstruction: {
      role: 'system',
      parts: [{"text": `For example, you are a helpful customer service agent.`}]
    },
});

const generativeVisionModel = vertexAI.getGenerativeModel({
    model: visionModel,
});

const generativeModelPreview = vertexAI.preview.getGenerativeModel({
    model: textModel,
});

async function generateContent() {
  const request = {
    contents: [{role: 'user', parts: [{text: 'How are you doing today?'}]}],
  };
  const result = await generativeModel.generateContent(request);
  const response = result.response;
  console.log('Response: ', JSON.stringify(response));
};

generateContent();

js-genai

Why so many packages, but still no way to call Agent engine agent

import {GoogleGenAI} from '@google/genai';
const GEMINI_API_KEY = process.env.GEMINI_API_KEY;

const ai = new GoogleGenAI({apiKey: GEMINI_API_KEY});

async function main() {
  const response = await ai.models.generateContent({
    model: 'gemini-2.0-flash-001',
    contents: 'Why is the sky blue?',
  });
  console.log(response.text);
}

main();

PredictionServiceClient

The endpoint is similar to my agent endpoint

import { PredictionServiceClient } from '@google-cloud/aiplatform';

async function predict() {
  const client = new PredictionServiceClient({keyFile: '<service account key file>'});
  const endpoint = `projects/<project-id>/locations/us-central1/reasoningEngines/<endpoint>`;

  const instances = [{
  class_method: "query",
  input: {
    input: "What is the exchange rate from US dollars to Swedish Krona today?"
  }
  }];

  try {
    const response = await client.predict({
      endpoint,
      instances,
    });

    // Log the prediction results correctly
    console.log(`Prediction results: ${response}`);
  } catch (error) {
    console.error('Error during prediction:', error);
  }
}

predict().catch(console.error);

error

'Request message serialization failure: .google.cloud.aiplatform.v1.PredictRequest.instances: array expected',

Doubt that PredictionServiceClient is for traditional machine learning which accepts array of instances.

Appendix

model discontinuation

Overview of Firebase-related service accounts

Vertex AI access control with IAM | Google Cloud

First Look: Exploring the Firebase CLI's Experimental MCP Server