Cloud-native microservice

Fragments

An API for text and image fragments that stored one original and converted supported formats on read.

5 min read

Glowing orange cube and connected fragments on a dark background

Supporting text and images can leave clients with a separate workflow for each format. I built Fragments as a coursework project to address that problem with one authenticated API for creating and retrieving content.

The key decision was to keep the original rather than save a copy of every conversion. Clients could request a supported representation later, while the stored fragment stayed the source of truth.

Create and read a fragment

  1. 01

    Authenticate

    Cognito validated each request and the API derived the owner's storage key.

  2. 02

    Create

    The API received text or image bytes with a supported content type.

  3. 03

    Store

    S3 held the original bytes and DynamoDB indexed the fragment metadata.

  4. 04

    Retrieve

    A later read checked owner-scoped metadata before loading the original bytes.

  5. 05

    Convert

    If requested, the API produced a supported format in memory.

  6. 06

    Respond

    The caller received the original or converted payload.

Deployment path

  1. 1GitHub Actions checks
  2. 2Docker image to ECR
  3. 3ECS service rollout

The release replaced the running container image on ECS Fargate. Fragment metadata and original bytes remained in DynamoDB and S3.

Architecture

The browser and AWS services had distinct jobs. Cognito provided identity, the load balancer routed API traffic, and ECS Fargate ran the application. The API coordinated owner-scoped metadata, original content, and logs.

Sign-in and API request

5 connected stepsShow
  1. 01

    fragments-ui

    Browser client

    signed in
  2. 02

    Amazon Cognito

    Authenticated the caller

    returned token
  3. 03

    fragments-ui with token

    Sent the API request

    bearer token
  4. 04

    Application Load Balancer

    Routed API traffic

    forwarded request
  5. 05

    ECS Fargate

    Ran the Fragments API

From the Fragments service

3 service handoffsShow

  • Fragments API on ECS

    Queried and updated owner-scoped records

    DynamoDB

    Fragment IDs, types, sizes, and timestamps

  • Fragments API on ECS

    Read and wrote original bytes

    S3

    Original text and image content

  • Fragments API on ECS

    Emitted application logs

    CloudWatch Logs

    Runtime logging

What it does

Managed text and images, then converted supported formats on read.

The API supported create, read, update, and delete operations for plain text, Markdown, HTML, JSON, CSV, YAML, PNG, JPEG, and WebP fragments.

A read could return the original or a supported conversion. For example, Markdown could become HTML or plain text, while a JPEG could become PNG, WebP, or AVIF. The conversion happened in memory and did not alter the saved original.

Storage and state

DynamoDB indexed ownership and metadata. S3 kept the original bytes.

Each DynamoDB record contained the fragment ID, content type, byte size, timestamps, and a SHA-256 hash of the owner's email. This supported owner-scoped queries without storing the email address in the record. S3 stored the content bytes separately.

On create, the API wrote the original to S3 and its metadata to DynamoDB. A read found the owner's record before loading the payload. Converted outputs were returned to the caller without creating more stored versions.

Security and access

Cognito verified identity before owner-scoped reads and writes.

The API accepted Basic credentials and bearer tokens and validated them through Cognito. It derived the owner hash used in storage queries after authentication, leaving password, session, and token management to Cognito.

Every read and write was scoped to the verified owner. A caller could not use a fragment ID to access another user's content.

Delivery and operations

GitHub Actions verified the service before Docker images reached ECS.

GitHub Actions ran tests, built the Node.js service into a Docker image, pushed it to ECR, and updated the ECS service. The application ran on ECS Fargate behind an Application Load Balancer.

The checks ran before image publication and rollout. A failed check stopped the release path rather than publishing an unverified image.

Testing and verification

Jest covered request handling, ownership, storage, and conversions.

The Jest suite covered successful requests, authentication failures, unsupported content types, missing fragments, invalid conversions, and malformed inputs. It exercised parsing, owner scoping, metadata consistency, object storage, and conversion behavior.

Local development and CI ran the same suite, so the release check matched the tests used while building the service.

Decisions and lessons

One stored original simplified updates but made reads do conversion work.

Keeping one original avoided extra stored formats and the invalidation work they would create after updates. The tradeoff was doing conversion work again when a client requested a derived format.

Separating metadata from bytes made each service's job clear, but left the API responsible for coordinating writes across DynamoDB and S3. The shared Jest suite and release checks made that coordination part of the normal development workflow.

Tech stack

Express.jsNext.jsAWS S3AWS DynamoDBAWS ECSAWS ECRAWS CognitoAWS CloudWatchAWS ALBDockerAnsibleGitHub ActionsJest
← Back to projects