feat: automatic nested stack partitioning for large schemas by praneetap · Pull Request #3437 · aws-amplify/amplify-category-api

praneetap · 2026-03-18T13:19:06Z

Automatic Nested Stack Partitioning for Large Schemas

Summary

Implements automatic partitioning of AppSync resolvers across multiple nested CloudFormation stacks to solve the 1MB template size limit issue affecting Amplify Gen 2 customers with large schemas.

Fixes:

Template may not exceed 1000000 bytes in size #2153 - Template size limit exceeded
Feature Request: Ability to Disable Autogenerated GraphQL Queries and Mutations for Models in Gen 2 #2559 - Large schema deployment failure
500 Number of resources limit and 1000000 Template size limit #2550 - CloudFormation 1MB limit
amplify-data#424 - Schema size issues

Problem

Customers with large schemas (100+ types, many custom resolvers) encounter CloudFormation's 1MB template size limit during deployment:

❌ Template may not exceed 1000000 bytes in size.
   Current size: 1,234,567 bytes

Current workarounds require:

Manual schema splitting (complex, breaks existing code)
Migrating off Amplify (losing platform benefits)
Removing features to reduce schema size

Solution

Automatic partitioning of AppSync resolvers across multiple nested stacks:

Root Stack
  ├─ DataPrimary (687 KB) - API, schema, tables, data sources
  ├─ DataResolvers0 (512 KB) - Resolvers 1-200
  └─ DataResolvers1 (498 KB) - Resolvers 201-347

Each nested stack has its own 1MB limit, effectively multiplying available template space.

Why This Architecture?

We keep tables and data sources in the Primary stack (not separate stacks) to avoid the 200 output limit bottleneck:

Our solution:

Primary Stack (API + tables + data sources)
- All resources accessible in parent scope
- Resolvers reference tables/sources by name via Fn::GetAtt
- No cross-stack imports needed
- No output limit pressure
Resolver Stacks (only resolvers)
- Reference API from parent scope
- No exports needed
- Can scale to many stacks without hitting limits

CloudFormation Limits Addressed

Limit	Per Stack	Our Threshold	Strategy
Template Size	1MB	750KB	Primary concern - distribute resolvers across stacks
Resources	500	450	Max 200 resolvers per stack = ~250 resources (safe)
Outputs	200	150	Keep tables/datasources in primary (no cross-stack imports)
Parameters	200	N/A	Resolvers reference API by name (no parameters needed)
Nested Stacks	No hard limit	<10 stacks	Practical operational limit

Key Features

1. Opt-In for CDK Users

import { AmplifyGraphqlApi } from '@aws-amplify/graphql-api-construct';

new AmplifyGraphqlApi(stack, 'api', {
  definition,
  authorizationModes,
  enableAutoPartitioning: true, // Opt-in
  partitioningConfig: {
    maxResolversPerStack: 150,
  },
});

2. Intelligent Distribution

Primary resources (API, tables, data sources) → Dedicated stack
Resolvers → Distributed across overflow stacks
Related resolvers (same GraphQL type) → Grouped together
Threshold-based → New stack created when approaching limit

3. Configurable

new AmplifyGraphqlApi(stack, 'api', {
  definition,
  authorizationModes,
  enableAutoPartitioning: true,
  partitioningConfig: {
    maxResolversPerStack: 150,    // More conservative
    stackSizeThreshold: 600000,   // Earlier split (600KB)
    groupRelatedResolvers: true,  // Keep types together
    maxCrossStackReferences: 100, // Output limit threshold
  },
});

4. CDK Context Support

cdk deploy --context amplify-data-auto-partition=true

Implementation Details

Core Changes

1. New `PartitioningNestedStackProvider` Class

packages/amplify-graphql-api-construct/src/internal/partitioning-nested-stack-provider.ts (416 lines)

Intelligent nested stack provider that:

Categorizes resources (primary, resolvers)
Tracks stack capacity (resolver count, estimated size, resource count, output count)
Routes resources to appropriate stacks
Creates overflow stacks as needed
Groups related resolvers for efficiency
Validates CloudFormation limits with warnings

2. Modified `AmplifyGraphqlApi` Constructor

packages/amplify-graphql-api-construct/src/amplify-graphql-api.ts

Accepts enableAutoPartitioning and partitioningConfig props
Uses PartitioningNestedStackProvider when enabled
Falls back to original single-stack behavior when disabled
Logs partitioning statistics with warnings

3. Extended Type Definitions

packages/amplify-graphql-api-construct/src/types.ts

Added:

PartitioningConfig interface with full documentation
enableAutoPartitioning?: boolean to AmplifyGraphqlApiProps
partitioningConfig?: PartitioningConfig to AmplifyGraphqlApiProps

4. Public API Exports

packages/amplify-graphql-api-construct/src/index.ts

Exported PartitioningConfig type for external consumers

Testing

Unit Tests (17 test cases)

src/__tests__/internal/partitioning-nested-stack-provider.test.ts

✅ Resource categorization (API, tables, datasources, resolvers)
✅ Resolver distribution and overflow behavior
✅ Capacity management (resolver count, size, resources)
✅ Statistics and reporting
✅ Limit validation (throws at 500 resources, 200 outputs)
✅ Stack naming conventions
✅ GraphQL type extraction (Query, Mutation, Subscription, custom types)
✅ Configuration defaults and custom values
✅ Edge cases (empty schema, single resolver, 1000+ resolvers)

Integration Tests (15 test cases)

src/__tests__/__functional__/partitioning.test.ts

✅ Single stack when partitioning disabled
✅ Multiple stacks when partitioning enabled
✅ Large schemas (50-100 models)
✅ Custom configuration respect
✅ CDK context enablement
✅ Tables/datasources in primary stack verification
✅ Backwards compatibility with single-stack behavior
✅ API properties preservation (apiId, graphqlUrl, realtimeUrl)
✅ Edge cases (no models, custom queries/mutations, very large schemas)
✅ Configuration validation

Migration Guide

For CDK Users (Opt-In)

// Before - single stack (may hit 1MB limit)
new AmplifyGraphqlApi(stack, 'api', {
  definition,
  authorizationModes,
});

// After - partitioned stacks (solves limit)
new AmplifyGraphqlApi(stack, 'api', {
  definition,
  authorizationModes,
  enableAutoPartitioning: true,
});

For Existing Deployments

No breaking changes - partitioning is opt-in via enableAutoPartitioning: true or CDK context flag.

Stack References

If you reference stacks by name:

// ❌ May break if you reference stack directly
const dataStack = backend.data.stack;

// ✅ Better: Reference resources, not stacks
const apiId = backend.data.resources.graphqlApi.apiId;

Performance Impact

Deployment Time

Small schemas: No change (single stack)
Large schemas: +10-30 seconds (multiple stacks deploy sequentially)

Trade-off: Slightly longer deployment vs. successful deployment

Runtime Performance

No impact - All resources resolve to same AppSync API
Resolver execution identical to single-stack deployment

Metrics & Monitoring

Console output shows partitioning statistics:

[Amplify Data] Partitioned into 3 nested stacks:
  - Primary stack resources: 250
  - Resolver stacks: 2
  - Total resolvers: 347
  - Avg resolvers per stack: 173
  - Max resources in any stack: 250/500

Warnings when approaching limits:

[Amplify Data] Warnings:
  - Stack DataPrimary near resource limit: 470/500

Checklist

Related PRs

Blocks: aws-amplify/amplify-backend#[PR-NUMBER] (backend-data integration)

Testing Instructions

Test Case 1: Small Schema (No Partitioning)

git checkout feature/nested-stack-partitioning
cd packages/amplify-graphql-api-construct
npm test -- partitioning-nested-stack-provider.test.ts

Test Case 2: Integration Tests

npm test -- partitioning.test.ts

Test Case 3: Manual CDK Test

import { AmplifyGraphqlApi } from '@aws-amplify/graphql-api-construct';

const api = new AmplifyGraphqlApi(stack, 'TestApi', {
  definition: AmplifyGraphqlDefinition.fromString(/* large schema */),
  authorizationModes: { apiKeyConfig: { expires: cdk.Duration.days(7) } },
  enableAutoPartitioning: true,
});

// Deploy and verify multiple stacks created in CloudFormation console

Customer Impact

Before This PR:

Large schemas → deployment failure
Manual workarounds required
Customers migrate off Amplify

After This PR:

Large schemas → automatic success (when enabled)
Opt-in for CDK users
Amplify handles complexity

This unblocks dozens of customers currently stuck at template size limits.

Questions for Reviewers

Default behavior: Currently enableAutoPartitioning defaults to false (opt-in). Should we make it true by default in a future major version?
Stack naming: Names DataPrimary, DataResolvers0, DataResolvers1 - are these acceptable?
Resolver grouping: Currently groups related resolvers (same type) together. Any concerns?
Export limits: Not currently tracking exports (200 limit). Our architecture avoids exports, but should we add explicit tracking?
Template size estimation: Currently uses heuristic (3KB per resolver). Should we implement actual template synthesis for accurate measurement in a future iteration?

Implements automatic partitioning of AppSync resolvers across multiple nested CloudFormation stacks to solve the 1MB template size limit. **What Changed:** - Added PartitioningNestedStackProvider for intelligent resource distribution - Extended AmplifyGraphqlApiProps with enableAutoPartitioning and partitioningConfig - Added comprehensive unit and integration tests **How It Works:** - PRIMARY stack: API + Tables + DataSources (minimizes cross-stack refs) - RESOLVER stacks: Distributed resolvers (max 200 per stack) - Automatic overflow creation when capacity thresholds reached - Groups related resolvers by GraphQL type for efficiency **CloudFormation Limits Managed:** - Template Size: 750KB threshold per stack - Resources: 450 resource threshold (500 limit) - Outputs: 150 output threshold (200 limit) - avoided via architecture - Parameters: Minimal usage (resolvers reference via parent scope) **Configuration:** - enableAutoPartitioning (default: false for opt-in) - partitioningConfig for advanced tuning - CDK context support: amplify-data-auto-partition **Tests:** - Unit tests for PartitioningNestedStackProvider (17 test cases) - Integration tests for AmplifyGraphqlApi (15 test cases) - Edge case coverage and configuration validation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Add ACTIVE_nested-stack-partitioning.md with PR #3437 analysis, correct integration point, and design constraints - Add Active Work Tracking section to AGENTS.md with lifecycle rules - Add Active Work section to .agent-docs/README.md index

…appings infrastructure - Add precise GitHub links for every claim about the call chain - Document existing stackMappings feature as the foundation to build on - Trace the full flow: AmplifyGraphqlApiProps → ExecuteTransformConfig → GraphQLTransform → TransformerContext → StackManager.resourceToStackMap - Key insight: we just need to compute stackMappings automatically, zero changes needed to StackManager or any transformer

eIBarto · 2026-03-30T13:15:12Z

This seems to be a long awaited change request. On behalf of myself and any other amplify user to have experienced this hard limit that stalls projects entirely I kindly ask to top up priority

RestingState · 2026-05-14T20:01:35Z

I'm getting TooManyResourcesInStack error for "data/amplifyData/FunctionDirectiveStack" 541 is greater than allowed maximum of 500: AWS::IAM::Role (90), AWS::IAM::Policy (90), AWS::AppSync::DataSource (90), AWS::AppSync::FunctionConfiguration (180), AWS::AppSync::Resolver (90), AWS::CDK::Metadata (1)

We are in a desperate need for this feature. We're blocked right now and don't know how to proceed. Please, resolve this issue asap, so that we can continue to build our application

RestingState · 2026-06-14T22:18:03Z

Are there any updates on this PR?

RestingState · 2026-06-29T13:32:36Z

Hi Amplify team,

Could someone please take a look at this PR when possible? The issue it addresses has been open for quite some time and continues to affect users in production environments.

A review, feedback, or an update on next steps would be greatly appreciated. Thank you for your time and for maintaining Amplify.

praneetap requested a review from a team as a code owner March 18, 2026 13:19

praneetap mentioned this pull request Mar 18, 2026

feat(backend-data): enable automatic nested stack partitioning aws-amplify/amplify-backend#3142

Open

8 tasks

jeffalBC approved these changes May 13, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: automatic nested stack partitioning for large schemas - #3437

feat: automatic nested stack partitioning for large schemas#3437
praneetap wants to merge 1 commit into
aws-amplify:mainfrom
praneetap:feature/nested-stack-partitioning

praneetap commented Mar 18, 2026

Uh oh!

eIBarto commented Mar 30, 2026

Uh oh!

RestingState commented May 14, 2026

Uh oh!

RestingState commented Jun 14, 2026

Uh oh!

RestingState commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

praneetap commented Mar 18, 2026

Automatic Nested Stack Partitioning for Large Schemas

Summary

Problem

Solution

Why This Architecture?

CloudFormation Limits Addressed

Key Features

1. Opt-In for CDK Users

2. Intelligent Distribution

3. Configurable

4. CDK Context Support

Implementation Details

Core Changes

1. New PartitioningNestedStackProvider Class

2. Modified AmplifyGraphqlApi Constructor

3. Extended Type Definitions

4. Public API Exports

Testing

Unit Tests (17 test cases)

Integration Tests (15 test cases)

Migration Guide

For CDK Users (Opt-In)

For Existing Deployments

Stack References

Performance Impact

Deployment Time

Runtime Performance

Metrics & Monitoring

Checklist

Related PRs

Testing Instructions

Test Case 1: Small Schema (No Partitioning)

Test Case 2: Integration Tests

Test Case 3: Manual CDK Test

Customer Impact

Questions for Reviewers

Uh oh!

eIBarto commented Mar 30, 2026

Uh oh!

RestingState commented May 14, 2026

Uh oh!

RestingState commented Jun 14, 2026

Uh oh!

RestingState commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

1. New `PartitioningNestedStackProvider` Class

2. Modified `AmplifyGraphqlApi` Constructor