Cloud Functions + PubSub のローカル開発環境をdocker-composeで構築

Cloud Function でhttpリクエストを受け取りPubSubを経由して別のFunctionを起動する、という機能を開発した際のローカル環境構築メモ。

Function用のイメージのビルド

Gemfile

source "https://rubygems.org"

gem "functions_framework", "~> 0.7"
gem "google-cloud-pubsub"

docker/function/Dockerfile

# Build: docker build -t function -f docker/function/Dockerfile .
FROM ruby:3.2.2
WORKDIR /function

COPY Gemfile /function/Gemfile
RUN bundle config --local without "development test" \
    && bundle install

ソースコードをvolumeで取り込んで bundle exec functions-framework-ruby --target xxxx でターゲットを切り替えることで同じイメージで http 側とサブスクライバ側のコンテナを起動する想定。

http側のFunction

protobuf形式のメッセージを送信したいので定義。

message.proto

syntax = "proto3";
option ruby_package = "Message::";

message Request {
  int64 id = 1;
  string body = 2;
}

下記コマンドでrbファイルを生成

mkdir lib
protoc --ruby_out=./lib ./message.proto

http.rb

require "functions_framework"
require "google/cloud/pubsub"
require_relative "lib/message_pb"

TOPIC = ENV["TOPIC_ID"]

FunctionsFramework.http "hello" do |request|
  FunctionsFramework.logger.info("hello http")
  pubsub = ENV["PUBSUB_EMULATOR_HOST"].nil? ? Google::Cloud::Pubsub.new : Google::Cloud::Pubsub.new(project_id: "emulator")
  topic = pubsub.topic(TOPIC)
  message = Message::Message.new(id: 123, body: "hello!")
  message = Message::Message.encode(message)
  topic.publish(message)

  { "message" => "OK" }
end

イベント側のファンクション

event.rb

require "base64"
require "functions_framework"
require_relative "lib/message_pb"

FunctionsFramework.cloud_event "hello-event" do |event|
  FunctionsFramework.logger.info("hello event")

  message = Message::Message.decode(Base64.decode64(event.data["message"]["data"]))
  FunctionsFramework.logger.info("id: #{message.id}, body: #{message.body}")
end

PubSubエミュレータ

zenn.dev

この記事が良さそうなので参考にさせてもらう。

docker/pubsub/Dockerfile

# Build: docker build -t pubsub -f docker/pubsub/Dockerfile .
FROM gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators

RUN apt-get update && \
    apt-get install -y git python3-pip netcat && \
    git clone https://github.com/googleapis/python-pubsub.git

WORKDIR /python-pubsub/samples/snippets
RUN pip3 install virtualenv && \
    virtualenv env && \
    . env/bin/activate && \
    pip3 install -r requirements.txt

COPY ./docker/pubsub/entrypoint.sh ./
EXPOSE 8085
ENTRYPOINT ["./entrypoint.sh"]

docker/pubsub/entrypoint.sh

#!/bin/bash

set -em

export PUBSUB_PROJECT_ID=$PROJECT_ID
export PUBSUB_EMULATOR_HOST=0.0.0.0:8085

gcloud beta emulators pubsub start --project=$PUBSUB_PROJECT_ID --host-port=$PUBSUB_EMULATOR_HOST --quiet &

while ! nc -z localhost 8085; do
  sleep 0.1
done

. env/bin/activate
python3 publisher.py $PUBSUB_PROJECT_ID create $TOPIC_ID
python3 subscriber.py $PUBSUB_PROJECT_ID create-push $TOPIC_ID $SUBSCRIPTION_ID $PUSH_ENDPOINT

fg %1
chmod +x docker/pubsub/entrypoint.sh

を忘れずに

docker-compose でまとめる

docker-compose.yaml

version: '3.8'
services:
  pubsub:
    image: pubsub:latest
    restart: always
    environment:
      - PROJECT_ID=emulator
      - TOPIC_ID=event-topic
      - SUBSCRIPTION_ID=event-subscription
      - PUSH_ENDPOINT=http://event:8080
  http:
    image: function:latest
    environment:
      - TOPIC_ID=event-topic
      - PUBSUB_EMULATOR_HOST=pubsub:8085
      - PUBSUB_PROJECT_ID=emulator
    ports:
      - 8080:8080
    volumes:
      - .:/function
    entrypoint: ["bundle", "exec", "functions-framework-ruby", "--source=http.rb","--target=hello"]
  event:
    image: function:latest
    volumes:
      - .:/function
    entrypoint: ["bundle", "exec", "functions-framework-ruby", "--source=event.rb","--target=hello-event"]

docker-compose up で起動

これで、http://localhost:8080/ にアクセスすると http側のfunctionが呼ばれ、pubsubを経由してイベント側のfunctionが呼ばれることがログから確認できる