Generate image captions using Hugging Face
Use the Hugging Face Inference API to make calls to 100,000+ Machine Learning models from Supabase Edge Functions.
We can combine Hugging Face with Supabase Storage and Database Webhooks to automatically caption for any image we upload to a storage bucket.
Hugging Face is the collaboration platform for the machine learning community.
Huggingface.js provides a convenient way to make calls to 100,000+ Machine Learning models, making it easy to incorporate AI functionality into your Supabase Edge Functions .
Open your Supabase project dashboard or create a new project .
Create a new bucket called images
.
Generate TypeScript types from remote Database.
Create a new Database table called image_caption
.
Create id
column of type uuid
which references storage.objects.id
.
Create a caption
column of type text
.
Regenerate TypeScript types to include new image_caption
table.
Deploy the function to Supabase: supabase functions deploy huggingface-image-captioning
.
Create the Database Webhook in the Supabase Dashboard to trigger the huggingface-image-captioning
function anytime a record is added to the storage.objects
table.
To generate the types.ts file for the storage and public schemas, run the following command in the terminal:
supabase gen types typescript --project-id=your-project-ref --schema=storage,public > supabase/functions/huggingface-image-captioning/types.ts
Find the complete code on GitHub .
import { serve } from 'https://deno.land/std@0.168.0/http/server.ts'
import { HfInference } from 'https://esm.sh/@huggingface/inference@2.3.2'
import { createClient } from 'jsr:@supabase/supabase-js@2'
import { Database } from './types.ts'
console. log ( 'Hello from `huggingface-image-captioning` function!' )
const hf = new HfInference (Deno.env. get ( 'HUGGINGFACE_ACCESS_TOKEN' ))
type SoRecord = Database [ 'storage' ][ 'Tables' ][ 'objects' ][ 'Row' ]
interface WebhookPayload {
type : 'INSERT' | 'UPDATE' | 'DELETE'
old_record : null | SoRecord
const payload : WebhookPayload = await req. json ()
const soRecord = payload.record
const supabaseAdminClient = createClient < Database >(
// Supabase API URL - env var exported by default when deployed.
Deno.env. get ( 'SUPABASE_URL' ) ?? '' ,
// Supabase API SERVICE ROLE KEY - env var exported by default when deployed.
Deno.env. get ( 'SUPABASE_SERVICE_ROLE_KEY' ) ?? ''
// Construct image url from storage
const { data , error } = await supabaseAdminClient.storage
. from (soRecord.bucket_id ! )
. createSignedUrl (soRecord.path_tokens ! . join ( '/' ), 60 )
const { signedUrl } = data
// Run image captioning with Huggingface
const imgDesc = await hf. imageToText ({
data: await ( await fetch (signedUrl)). blob (),
model: 'nlpconnect/vit-gpt2-image-captioning' ,
// Store image caption in Database table
await supabaseAdminClient
. insert ({ id: soRecord.id ! , caption: imgDesc.generated_text })
return new Response ( 'ok' )