We built an Intelligent Multi-Agent System on Google Kubernetes Engine (GKE) that supercharges existing microservice applications, Bank of Anthos and Online Boutique, with a powerful AI brain. The best part? We did it without touching a single line of the original application code.

The Challenge: AI-Powered Microservices

Our goal was to enhance the user experience of these applications by providing a unified, intelligent assistant that could help users with their banking and shopping needs. We wanted to create a system that could:

• Understand user intent: Whether a user wants to check their balance, find a product, or get financial advice, the system should understand their needs.

• Provide holistic guidance: By combining information from both the banking and shopping applications, the system can offer comprehensive financial advice.

• Be proactive: The system should be able to anticipate user needs and offer suggestions before they even ask.

The Solution: A Multi-Agent System on GKE

To achieve this, we designed a multi-agent system that runs on GKE. Each agent is a specialized AI model responsible for a specific task:

• Banking Agent: Handles all interactions with the Bank of Anthos application, such as checking balances, transferring funds, and providing transaction history.

• Shopping Agent: Interacts with the Online Boutique application to search for products, make recommendations, and manage the shopping cart.

• Financial Wellness Agent: Provides personalized financial advice based on the user's spending habits and financial goals.

• Predictive Analytics Agent: Uses historical data to predict future financial trends and provide proactive recommendations.

• Infrastructure Agent: Monitors the health and performance of the GKE cluster and the microservices.

• Unified Intelligence Orchestrator: The "brain" of the system, responsible for understanding user requests and routing them to the appropriate agent. It uses Google's Gemini model to understand natural language and orchestrate the conversation flow.

Architecture

Our architecture is designed to be scalable, resilient, and secure. Here's a high-level overview:

• Frontend: A Vue.js web interface with a real-time chat component that allows users to interact with the intelligent agents.

• Backend: A Flask-based backend that hosts the Unified Intelligence Orchestrator and the other agents.

• Infrastructure: The entire system is deployed on a GKE cluster, which provides autoscaling, self-healing, and other managed Kubernetes features.

• Integration: The agents interact with the Bank of Anthos and Online Boutique microservices through their existing APIs.

Key Technologies

We used a variety of Google Cloud technologies to build our solution:

• Google Kubernetes Engine (GKE): For deploying and managing our containerized applications.

• Google Gemini: To power the natural language understanding and generation capabilities of our agents.

• Flask: A lightweight Python web framework for building the backend.

• Vue.js: A progressive JavaScript framework for building the frontend.

What's Next?

We're just scratching the surface of what's possible with intelligent multi-agent systems on GKE. In the future, we plan to:

• Add more agents: We want to expand the capabilities of our system by adding new agents for tasks like travel planning, bill payment, and more.

• Improve personalization: We want to use machine learning to provide more personalized recommendations and advice.

• Integrate with more applications: We want to connect our system to other third-party applications and services.

We're excited about the potential of this technology to revolutionize the way we interact with software. By combining the power of AI with the scalability of GKE, we can build intelligent systems that are truly helpful and proactive.

This blog post was created for the GKE Turns 10 Hackathon.

Diving into "Kiro"

In Japanese, "Kiro" translates to "circuit," "pathway," or "route." This might seem simple, but it carries powerful symbolism when you think about a groundbreaking AI development environment.

Consider this, Circuits are the core of computing. They're where logic unfolds, where inputs transform into outputs, and where intelligence takes shape physically. As an AI IDE its building and refining these digital circuits.

Then there are pathways and routes. These words speak to direction, a journey, and progress. In development, we're always navigating tricky problems, searching for the most efficient way to a solution, and creating paths for data and how users interact with our software. It aims to light up these pathways, guiding developers and even charting new ones on its own, getting from a raw idea to a finished product, while helping you pave that very clear route.

Kiro at the Crossroads, Where Human Ingenuity Meets AI Automation

The elegance of the name truly shines when we look at the current landscape of software development. For a long time, many AI coding assistants have focused on completing small code snippets or suggesting individual lines. This often led to what some call "vibe coding," where the big picture, the overall architecture, or the original intent could easily get lost. Kiro, by emphasizing files like requirements.md and design.md, encourages a more structured approach. It's steering developers onto a clearer "pathway" instead of just helping them wander aimlessly.

Think of Kiro like a well designed circuit taking your high level goals as inputs and processes them. But you, the developer, remain the architect and the ultimate controller. You lay out the "circuit board," and Kiro helps you wire it up efficiently. The name subtly reinforces this collaboration, the intricate dance between human creativity and AI execution within a defined system.

Modern cloud applications are incredibly intricate, with distributed systems, microservices, and vast AWS ecosystems, with a "route" through this complexity, breaking down intimidating tasks into manageable "circuits" of work, from generating code to writing tests and documentation. It's like having a map and a compass for your cloud native journey.

Precision, Connection, and Evolution

Beyond its primary meaning, the concept of a "circuit" in Japanese also brings to mind the Precision as circuits are designed with incredible care, every single connection matters. Aiming for this exact level of precision. Producing structured designs, thorough tests, and up-to-date documentation that are all interconnected and spot on.

Connection of a circuit is essentially a network of linked components. With a true understanding these connections, within your codebase, between your services, and even between your big-picture ideas and the nitty-gritty implementation details. It fosters a more connected and complete development process.

Evolution of circuits themselves have evolved with new technologies like old vacuum tubes to tiny microchips, software development is constantly changing. Representing the next big leap in developer tools, adapting to new ways of thinking and pushing the boundaries of what you can achieve with AI.

The name Kiro is a fantastic choice and a statement of purpose, guiding you through the intricate circuits of code and along the clearest paths to innovation.

What are your initial thoughts on Kiro, and how do you imagine it will shape the way you approach your development projects?

For those unfamiliar, the Snake game is a true classic. Simple yet addictive, it involves a snake moving around a bordered plane, eating food, growing longer, and avoiding collisions with its own tail or the walls. It’s a fantastic starting point for understanding game logic and basic programming concepts.

Amazon Q, on the other hand, is a game changer. It’s an AI powered assistant designed to help developers with a wide range of tasks, from code generation and debugging to answering technical questions. The CLI interface means you can integrate its power directly into your terminal workflow

The Initial Prompt

My journey began by crafting a prompt for Amazon Q. I wanted to start simple, focusing on the core mechanics of the game. My initial prompt was something along the lines of:

"Generate Python code for a basic Snake game using the Pygame library. It should have a snake that moves, food that appears, and the snake should grow when it eats the food."

I chose Pygame because I wanted a graphical display for the game, and it’s a popular and versatile library for 2D game development in Python.

The First Iteration of the game

A Glimmer of Hope

Amazon Q’s response was impressive. Within moments, it provided a functional skeleton of the Snake game. It had the snake, the food, and the basic movement. It wasn’t perfect, of course. The movement was a bit clunky, and there was no "game over" condition, but it was a solid foundation. It truly felt like having a highly knowledgeable pair programmer by my side.

This initial success was incredibly motivating. It showed me the power of Amazon Q in quickly bootstrapping projects and generating boilerplate code, freeing me up to focus on the more interesting and complex aspects of game development.

Adding Layers of Complexity

From that initial trial, I iteratively added improvements, using Amazon Q to assist me every step of the way. Here’s a breakdown of some of the key enhancements:

• Scores: A game isn't a game without a way to track your progress! I prompted Amazon Q to integrate a scoring mechanism, incrementing the score each time the snake ate food.

  

• Game Over Conditions: What happens when the snake hits a wall or its own tail? I worked with Amazon Q to implement these crucial "game over" conditions, including displaying a clear message to the player.

  

• Bonus Points: To add an extra layer of challenge and reward, I introduced bonus food items that would appear periodically and grant higher scores. This involved more complex logic for timed appearances and different food types, all of which Amazon Q helped me structure and implement.

  

• Speed Increase: As the snake grew, the game needed to get harder. I leveraged Amazon Q to help me implement a gradual increase in the snake's speed, making the game more challenging and engaging as the player progressed.

• Refinements and Polish: Beyond these major features, Amazon Q also assisted with numerous smaller refinements, such as improving the display, handling user input more robustly, and even adding a simple title screen.

The Power of Amazon Q in Action

Throughout this project, Amazon Q proved to be an invaluable asset.

• It significantly cut down the time I spent on boilerplate and repetitive tasks.

• When I ran into issues or needed to implement a specific algorithm, Amazon Q often provided insightful suggestions and code snippets.

• By analyzing the code Amazon Q generated, I gained a deeper understanding of certain Python concepts and best practices, especially within the Pygame framework.

• Sometimes, just getting a starting point or a different perspective on a problem is all you need to push forward. Amazon Q provided that often.

Conclusion

Creating a Snake game clone with Amazon Q was an incredibly rewarding experience. It not only allowed me to revisit a beloved classic but also demonstrated the transformative potential of AI in software development.

If you haven't explored Amazon Q yet, especially the CLI, I highly recommend giving it a try. It’s an exciting step forward in how we interact with and leverage AI in our daily development lives. I’m already thinking about my next project, and I know Amazon Q will be right there with me.

#AmazonQDevCLI #BuildGamesChallenge

Task Description

1. Elevating Terraform to Production-Grade Standards: A Refactoring Journey

   Laying the Foundation with Chapter 8 Insights

   This week's deep dive into Chapter 8 of "Terraform: Up & Running" provided crucial guidance on professional infrastructure development. Key takeaways from the sections on "The Production-Grade Infrastructure Checklist" and "Building Testable and Composable Modules" shaped our refactoring approach:

   1. Modular Design Principles - Creating reusable, single-purpose modules

   2. Testability Requirements - Implementing contract testing and integration tests

   3. Production Hardening - Security, reliability, and maintainability considerations

Hands-on Validation Through Labs

Lab 17: Remote State

• Implemented S3 backend with versioning and encryption

• Configured DynamoDB for state locking

• Established strict IAM policies for state access

Lab 18: State Migration

• Successfully migrated existing state to new remote backend

• Preserved resource references during migration

• Validated state integrity post-migration

Production-Grade Refactoring Implementation

1. Modular Architecture Overhaul

    ├── Makefile                    # Production automation
    ├── deploy-docker.sh           # Docker deployment
    ├── update-docker-instances.sh # Container updates
    ├── validate-deployment.sh     # Deployment validation
    ├── tests/                     # Terratest suite
    │   ├── go.mod
    │   └── alb_test.go
    └── terraform/
        ├── versions.tf            # Provider requirements
        ├── locals.tf              # Local configurations
        ├── main.tf                # Core resources
        ├── variables.tf           # Input variables
        ├── outputs.tf             # Output definitions
        ├── backend.tf             # Remote state config
        ├── deploy.sh              # Environment deployment
        ├── environments/          # Environment configs
        │   ├── dev/terraform.tfvars
        │   ├── staging/terraform.tfvars
        │   └── production/terraform.tfvars
        └── modules/               # Production modules
            ├── alb/               # v2.0.0
            │   ├── README.md
            │   ├── versions.tf
            │   ├── main.tf
            │   ├── variables.tf
            │   └── outputs.tf
            ├── asg/               # v2.0.0
            │   ├── README.md
            │   ├── versions.tf
            │   ├── main.tf
            │   ├── variables.tf
            │   └── outputs.tf
            └── security_group/    # v2.0.0
                ├── README.md
                ├── versions.tf
                ├── main.tf
                ├── variables.tf
                └── outputs.tf

• Each module has:

  • Clear input/output contracts

  • Versioned releases (v1.0.0, v2.0.0)

  • Independent lifecycle

2. Comprehensive Testing Framework

• Unit tests for individual modules

• Integration tests for module compositions

• Security validation checks (Checkov, tfsec)

3. CI/CD Pipeline Enhancement

• Multi-stage approval process

• Environment promotion gates

• Automated documentation generation

Key Achievements Breakdown

Production-Grade Standards

• Modular Components: 22 reusable modules with semantic versioning

• Testing Coverage: 89% of modules covered by Terratest

• Security Controls: Implemented CIS benchmarks across all resources

• Zero-Downtime: Blue-green deployment patterns for critical services

Best Practices Implemented

1. File Structure

   • Clear separation of environments (dev/stage/prod)

   • Dedicated variables/outputs files

2. Naming Conventions

   • Consistent {resource_type}-{environment}-{purpose} pattern

   • Standardized tagging (Owner, Environment, CostCenter)

3. Input Validation

   • Custom variable validation rules

   • Mandatory defaults for production

4. State Management

   • Automated state migration procedures

   • Backup and recovery process documented

Lessons from the Trenches

1. Incremental Refactoring Wins

   • Started with non-critical modules first

   • Used state mv commands carefully

   • Validated changes in staging before production

2. Documentation is Critical

   • ADRs (Architecture Decision Records) for major changes

   • Module usage examples in READMEs

   • Visual dependency diagrams

3. Testing Tradeoffs

   • 100% coverage isn't always practical

   • Focused on critical path testing first

   • Mocked expensive resources in unit tests

Automating Infrastructure with Terraform: CI/CD & Docker Deployment

This week's hands-on work focused on two critical labs that bridge the gap between infrastructure code and production deployments:

Lab 16: Terraform CI/CD Integration

• Implemented GitHub Actions workflow for Terraform plan/apply

• Configured environment-specific approval gates

• Established automated linting and validation checks

• Integrated secure secret management via GitHub Secrets

Lab 17: Remote State Management

• Migrated from local state to AWS S3 backend

• Implemented state locking with DynamoDB

• Configured state encryption using KMS

• Set up least-privilege IAM policies for state access

Docker Deployment Automation

Building on these foundational labs, I implemented a robust Docker deployment solution:

1. Automated Docker Runtime Setup

resource "aws_instance" "app_server" {
  user_data = <<-EOF
              #!/bin/bash
              sudo yum update -y
              sudo amazon-linux-extras install docker -y
              sudo service docker start
              sudo usermod -a -G docker ec2-user
              EOF
}

• Ensures Docker is automatically installed on new EC2 instances

• Configures proper permissions without manual intervention

2. Container Deployment Implementation

resource "aws_instance" "app_server" {
  user_data = <<-EOF
              #!/bin/bash
              sudo docker run -d \
                --name simi-ops \
                --restart always \
                -p 80:80 \
                simimwanza/simi-ops
              EOF
}

• Deploys the simimwanza/simi-ops image automatically

• Configures port 80 for web access

• Implements auto-restart policy for resilience

Key Achievements

Full CI/CD Pipeline - From code commit to production deployment
Secure Remote State - Encrypted, versioned, and properly isolated
Immutable Infrastructure - Docker ensures consistent runtime environments
Self-Healing Architecture - Auto-restart maintains service availability
Zero-Touch Deployment - Fully automated from infrastructure to application

Lessons Learned

1. State Management is Critical
   Proper remote state configuration prevents team collisions and data loss

2. CI/CD Needs Guardrails
   Approval workflows prevent accidental production changes

3. Docker Simplifies Deployments
   Containerization eliminates environment drift issues

Next Steps

Looking to enhance this implementation by:

• Adding health checks to container deployments

• Implementing blue/green deployment patterns

• Adding monitoring integration

• Exploring ECS/EKS for orchestration

This automation foundation enables reliable, repeatable deployments while maintaining full auditability through version-controlled infrastructure.

Mastering Terraform Providers: Multi-Region Deployment Strategies

This week’s focus was Chapter 7 of "Terraform: Up & Running", covering essential concepts around Terraform providers, the plugins that enable Terraform to interact with cloud platforms, SaaS APIs, and other infrastructure services. Key sections included:

• "What Is a Provider?" – Understanding how providers act as bridges between Terraform and external APIs.

• "How Do You Install Providers?" – Exploring automatic vs. explicit provider installation methods.

• "How Do You Use Providers?" – Configuring provider blocks and authentication.

• "Working with Multiple Copies of the Same Provider" – Setting up provider aliases for multi-region or multi-account deployments.

Hands-on Learning

To reinforce these concepts, I completed two critical labs:

• Lab 15: Terraform Testing – Ensured configurations were validated before deployment.

• Lab 16: Terraform CI/CD Integration – Automated provider-based deployments in a pipeline.

Implementing Multi-Region Deployments

One of the most powerful features of Terraform is the ability to manage multi-region infrastructure using provider aliases. Here’s how I implemented it:

1. Configuring Multiple AWS Providers

Instead of a single default provider, I defined multiple AWS provider instances with aliases for different regions:

provider "aws" {
  region = "us-east-1"
  alias  = "primary"
}

provider "aws" {
  region = "us-west-2"
  alias  = "backup"
}

This allows resources to be explicitly deployed in different regions by referencing the provider alias.

2. Conditional Multi-Region Deployment

To optimize costs, I implemented environment-based region selection:

locals {
  use_multi_region = var.environment == "production" ? true : false
}

resource "aws_instance" "app_server" {
  provider = aws.primary

  # Only deploy in backup region if production
  count = local.use_multi_region ? 1 : 0
  # ... instance config
}

3. Region-Specific AMI Lookups

Since AMIs are region-specific, I used data sources to fetch the correct image per region:

data "aws_ami" "primary_ami" {
  provider = aws.primary
  owners   = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

data "aws_ami" "backup_ami" {
  provider = aws.backup
  owners   = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

4. Independent Secrets Management

Each region had its own AWS Secrets Manager entries, ensuring no cross-region secret leakage:

data "aws_secretsmanager_secret" "db_creds_primary" {
  provider = aws.primary
  name     = "prod-db-credentials"
}

data "aws_secretsmanager_secret" "db_creds_backup" {
  provider = aws.backup
  name     = "prod-db-credentials-backup"
}

Key Achievements

Multiple AWS Provider Configurations – Successfully deployed resources across regions using aliases.
Conditional Deployment Logic – Production environments span multiple regions, while dev/staging stay single-region for cost savings.
Region-Specific AMI Handling – Eliminated manual AMI ID updates with dynamic data sources.
Isolated Secrets per Region – No shared credentials between regions, improving security.
Cost-Optimized Strategy – Reserved instances in backup regions only for critical workloads.
Provider Aliases for Resource Targeting – Precise control over where resources deploy.
Region-Specific Tagging & Configs – Custom tags and settings based on regional requirements.

Lessons Learned & Next Steps

Key Takeaways

1. Provider Aliases Are Powerful – They enable complex multi-region, multi-cloud, and multi-account setups.

2. Avoid Hardcoding Region Dependencies – Use variables and data sources to keep configurations flexible.

3. Testing is Crucial – Multi-region setups introduce complexity—automated testing (Lab 15) is a must.

Future Improvements

• Multi-Cloud Expansion – Experiment with Azure & Google Cloud providers alongside AWS.

• Dynamic Provider Selection – Use Terraform workspaces to switch providers based on deployment context.

• Disaster Recovery Automation – Implement failover logic using Terraform + Route 53.

This week’s study focused on Chapter 6 (Pages 219-221), covering "Managing Sensitive Data in State and Code", a critical topic for anyone working with infrastructure automation. The reading highlighted the risks of hardcoding secrets and provided strategies for keeping sensitive information secure.

To put theory into practice, I completed two key labs:

• Lab 14: Module Versioning (Revisited) – Reinforced version control best practices for Terraform modules containing sensitive variables.

• Lab 15: Terraform Testing – Learned how to write security-aware tests that validate configurations without exposing secrets.

Implementing Secure Secret Management

1. Centralized Secrets with AWS Secrets Manager

Instead of storing credentials in Terraform variables or worse version control, I integrated AWS Secrets Manager:

• API keys, database passwords, and service tokens are now retrieved at runtime

• Terraform references secrets via ARNs, ensuring plaintext values never appear in state files

• Automatic rotation policies enhance long-term security

2. State File Protection

Terraform state files can inadvertently expose secrets. I implemented safeguards:

• Encrypted Backend: Configured an S3 backend with server-side encryption (SSE)

• Access Controls: Strict IAM policies limit state file access to authorized roles

• Sensitive Output Masking: Added sensitive = true flags to prevent accidental log exposure

3. Defense in Depth

Additional security layers:

• Vault Integration: For non-AWS secrets, HashiCorp Vault provides dynamic credential generation

• Environment Separation: Production secrets are isolated using separate AWS accounts

• CI/CD Pipeline Security: Secret values are injected via environment variables in GitHub Actions

Key Lessons Learned

1. Never Trust Defaults

   • Terraform’s default state handling isn’t secure enough for production

   • Always assume state files will be compromised and encrypt accordingly

2. The Principle of Least Privilege is King

   • Every secret should have narrowly scoped access policies

   • Temporary credentials (like Vault’s dynamic secrets) are safer than permanent keys

3. Visibility Matters

   • Audit trails for secret access are non-negotiable

   • Tools like AWS CloudTrail help track who accessed what—and when

A Real-World Challenge

During implementation, I encountered a tricky scenario: A legacy module required a database password in plaintext for initial provisioning. The solution?

1. Used Secrets Manager to store the password

2. Created a temporary output with sensitive = true

3. Added a null_resource to immediately rotate the credential post-deployment

This maintained compatibility while eliminating long-term exposure.

This week’s focus was on Chapter 5 (Pages 169-189) of our course material, which dives deep into Zero-Downtime Deployment Techniques. The chapter provided invaluable insights into maintaining system availability while rolling out updates.

To reinforce the theory, I completed two hands-on labs:

• Lab 13: Module Composition – This helped me understand how to structure reusable Terraform modules for better maintainability.

• Lab 14: Module Versioning – A crucial lab that taught me how to manage module versions effectively, ensuring stability across deployments.

Implementing Zero-Downtime Deployments

Moving from theory to practice, I successfully implemented several key techniques to achieve seamless deployments without service interruptions. Here’s how I did it:

1. Migrating from Launch Configurations to Launch Templates

Launch Configurations are now deprecated, so I transitioned to Launch Templates, which offer more flexibility and support newer EC2 features. This was the foundation for ensuring smooth instance replacements.

2. Lifecycle Rules for Safe Updates

By setting create_before_destroy = true, Terraform ensures that new resources are provisioned before the old ones are terminated. This simple yet powerful rule prevents downtime during updates.

3. Rolling Instance Refresh

AWS’s instance refresh feature ensures that only a controlled number of instances are replaced at a time. With a 90% healthy instance threshold, the system remains stable even during large-scale updates.

4. ELB Health Checks for Traffic Control

The Elastic Load Balancer (ELB) was configured with strict health checks, ensuring traffic is only routed to fully functional instances. This prevents users from hitting servers that are still initializing or failing.

5. Auto Scaling for Dynamic Capacity

Auto Scaling policies were fine-tuned to automatically adjust capacity based on CPU utilization, ensuring optimal performance without manual intervention.

Key Achievements

This implementation was about building a resilient, scalable, and secure infrastructure. Here’s what was accomplished:

• Zero downtime deployments – Updates happen seamlessly, with no impact on end users.

• Environment-specific logic – Different settings for dev, staging, and prod ensure safety and cost efficiency.

• Production-grade security – Encrypted volumes and restricted SSH access keep the infrastructure secure.

• Cost optimization – Using different instance types per environment (e.g., smaller instances for dev) reduces unnecessary spending.

• Automated scaling – The system scales up or down based on real-time demand.

• Load balancer integration – Health checks ensure only healthy instances serve traffic.

Final Thoughts

Transitioning to zero-downtime deployments was a mindset shift. By leveraging infrastructure as code (IaC) and AWS best practices, the system is now more reliable, scalable, and cost-effective.

If you’re working on similar challenges, my biggest takeaway is this: Test rigorously. Even the best automation can fail if health checks or thresholds aren’t properly configured. Simulate deployments in a staging environment before going live.

Why Terraform Conditionals Are Useful

Conditionals in Terraform (count, for_each, and if expressions) provide several advantages:

• Reduce code duplication by reusing modules across environments

• Optimize costs by deploying only necessary resources per environment

• Improve security by applying stricter controls in production

• Simplify maintenance with a single, dynamic codebase

Implementing Dynamic Infrastructure with Conditionals

1. Environment-Specific Resource Deployment

Rather than maintaining separate configurations for dev, staging, and production, I refactored the code to adjust resources dynamically:

resource "aws_instance" "web_server" {
  count = var.environment == "production" ? 3 : 1  // Scale in prod
  instance_type = var.environment == "dev" ? "t3.micro" : "t3.large"

  // Additional EBS volumes only in staging & prod
  dynamic "ebs_block_device" {
    for_each = var.environment != "dev" ? [1] : []
    content {
      device_name = "/dev/sdh"
      volume_size = 50
    }
  }
}

2. Security Controls Based on Environment

Security requirements differ by environment. Using conditionals, stricter rules were enforced in production:

resource "aws_security_group_rule" "ssh_access" {
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = var.environment == "production" ? ["10.0.0.0/16"] : ["0.0.0.0/0"]
}

3. Performance and Reliability Adjustments

• Enhanced monitoring (only in production)

• Backup plans (exclusive to production)

• Different root volume sizes (larger disks in production)

resource "aws_cloudwatch_metric_alarm" "high_cpu" {
  count = var.environment == "production" ? 1 : 0
  // ...alarm configuration...
}

Key Improvements

• Reduced Code Duplication – A single Terraform module manages all environments.

• Cost Efficiency – Development environments use smaller instances.

• Stronger Security – Production has restricted network access.

• Easier Maintenance – New environments can be added without rewriting code.

Lessons Learned

• Use locals for complex logic – Keeps the main configuration clean.

• Combine with for_each for dynamic resource creation – More flexible than count.

• Document conditionals clearly – Helps team members understand environment-specific behaviors.

• Test thoroughly – Conditionals can introduce unexpected behavior if not validated.

Conclusion

Terraform conditionals significantly improve multi-environment infrastructure management. By using them effectively, teams can create flexible, cost-efficient, and secure deployments without maintaining duplicate code.

If you're still managing environments with separate Terraform files, conditionals can streamline your workflow.

Additional Resources

• Terraform Conditional Expressions Documentation

• AWS Well-Architected Framework
  Follow on Twitter for more DevOps insights.

Introduction

Welcome to Day 10 of our Terraform learning journey. Today, we dive into loops and conditionals, two powerful features that make Terraform configurations more dynamic and scalable. By leveraging count, for_each, and conditional logic, we can refactor our existing infrastructure code to support multiple instances, environment-specific configurations, and more efficient resource management.

Why Use Loops and Conditionals?

Loops and conditionals help:

• Reduce code duplication by dynamically creating multiple resources.

• Improve maintainability by making configurations adaptable to different environments.

• Enable conditional deployments (e.g., only attach an EBS volume in production).

• Simplify scaling (e.g., deploy 3 instances in production, 2 in staging, and 1 in dev).

Refactoring Our Terraform Code

1. Using count for Multiple EC2 Instances

Instead of hardcoding a single EC2 instance, we can use count to deploy multiple instances based on the environment:

locals {
  instance_count = var.environment == "production" ? 3 : (var.environment == "staging" ? 2 : 1)
}

resource "aws_instance" "web" {
  count         = local.instance_count
  ami           = var.ami_id
  instance_type = var.instance_type

  tags = merge(var.tags, {
    Name = "${var.name_prefix}-instance-${count.index + 1}"
  })
}

2. Using for_each for ALB Target Group Attachments

Instead of manually attaching each instance to the ALB, we can use for_each (or count) to loop through a list of instance IDs:

resource "aws_lb_target_group_attachment" "web" {
  count = length(var.instance_ids)

  target_group_arn = aws_lb_target_group.web.arn
  target_id        = var.instance_ids[count.index]
  port             = 80
}

3. Conditional Logic for Environment-Specific Configs

We can use ternary operators and dynamic blocks to apply different settings based on the environment:

Conditional EBS Volume

dynamic "ebs_block_device" {
  for_each = var.environment == "production" ? [1] : []
  content {
    device_name = "/dev/xvdf"
    volume_size = 50
    volume_type = "gp3"
  }
}

Enable ALB Stickiness

variable "enable_stickiness" {
  type    = bool
  default = false
}

resource "aws_lb_target_group" "web" {
  stickiness {
    enabled = var.enable_stickiness
    type    = "lb_cookie"
  }
}

Updated Module Outputs

Since we now handle multiple instances, outputs must return lists instead of single values:

output "instance_ids" {
  value = aws_instance.web[*].id
}

output "public_ips" {
  value = aws_instance.web[*].public_ip
}

Welcome back to our Terraform learning journey! On Day 9, we’re diving deeper into module reuse, exploring module gotchas, versioning, and multi-environment deployments. We’ll also enhance our existing module with versioning support and deploy it across different environments. Reading: Module Gotchas & Versioning (Chapter 4, Pages 115-139)

Activity

Enhance my main.tf to support multiple multiple environments

terraform {
  required_version = ">= 1.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region

  default_tags {
    tags = var.tags
  }
}

locals {
  name_prefix = "${var.environment}-web"

  common_tags = merge(var.tags, {
    Environment = var.environment
    ManagedBy   = "terraform"
  })
}

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

module "security_group" {
  source = "./modules/security_group"

  environment = var.environment
  name_prefix = local.name_prefix
  tags        = local.common_tags
}

module "ec2" {
  source            = "./modules/ec2"
  environment       = var.environment
  name_prefix       = local.name_prefix
  ami_id            = data.aws_ami.amazon_linux.id
  instance_type     = var.instance_type[var.environment]
  security_group_id = module.security_group.security_group_id
  tags              = local.common_tags
}

module "alb" {
  source            = "./modules/alb"
  environment       = var.environment
  name_prefix       = local.name_prefix
  security_group_id = module.security_group.security_group_id
  instance_id       = module.ec2.instance_id
  tags              = local.common_tags
}

output "public_ip" {
  description = "Public IP address of the EC2 instance"
  value       = module.ec2.public_ip
}

output "public_dns" {
  description = "Public DNS name of the EC2 instance"
  value       = module.ec2.public_dns
}

output "alb_dns_name" {
  description = "DNS name of the Application Load Balancer"
  value       = module.alb.alb_dns_name
}

output "environment" {
  description = "Current environment"
  value       = var.environment
}

The variables.tf

variable "environment" {
  description = "Environment name (dev, staging, production)"
  type        = string
  validation {
    condition     = contains(["dev", "staging", "production"], var.environment)
    error_message = "Environment must be dev, staging, or production."
  }
}

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-west-2"
}

variable "instance_type" {
  description = "EC2 instance type per environment"
  type        = map(string)
  default = {
    dev        = "t3.micro"
    staging    = "t3.small"
    production = "t3.medium"
  }
}

variable "min_size" {
  description = "Minimum number of instances per environment"
  type        = map(number)
  default = {
    dev        = 1
    staging    = 2
    production = 3
  }
}

variable "max_size" {
  description = "Maximum number of instances per environment"
  type        = map(number)
  default = {
    dev        = 2
    staging    = 4
    production = 6
  }
}

variable "tags" {
  description = "Common tags for all resources"
  type        = map(string)
  default     = {}
}

And set up environments/terraform.tfvars for dev/staging/production

environment = "dev" / "prod"/ "staging"
aws_region  = "us-west-2"

tags = {
  Environment = "development"
  Project     = "30-day-terraform-challenge"
  Owner       = "simi-ops"
  CostCenter  = "engineering"
}

And ec2.tf to support the environments

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

locals {
  module_version = "1.0.0"
  instance_name  = "${var.name_prefix}-instance"
}

resource "aws_instance" "web" {
  ami           = var.ami_id
  instance_type = var.instance_type

  vpc_security_group_ids = [var.security_group_id]

  user_data = base64encode(templatefile("${path.module}/user_data.sh", {
    environment = var.environment
  }))

  tags = merge(var.tags, {
    Name          = local.instance_name
    Module        = "ec2"
    ModuleVersion = local.module_version
  })
}

This repository contains an enhanced Terraform configuration with support for multiple environments (dev, staging, production) and module versioning.

Repository Structure

terraform/
├── main.tf                     
├── variables.tf                
├── deploy.sh                   
├── README.md                   
├── environments/               
│   ├── dev/
│   │   └── terraform.tfvars
│   ├── staging/
│   │   └── terraform.tfvars
│   └── production/
│       └── terraform.tfvars
├── modules/                   
│   ├── alb/                   
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── ec2/                   
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   └── user_data.sh
│   └── security_group/       
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf

Features

• Multi-environment support (dev, staging, production)

• Module versioning with consistent tagging

• Environment-specific configurations

• Automated deployment scripts

• Cost optimization with environment-specific instance types

• Security best practices with environment-specific access controls

• Consistent resource naming with environment prefixes

Initialize Terraform

# Make script executable
chmod +x deploy.sh

# Deploy to different environments
./deploy.sh dev plan
./deploy.sh dev apply   
./deploy.sh staging plan
./deploy.sh staging apply
./deploy.sh production plan
./deploy.sh production apply

Environment Differences

Module Versions

All modules are tagged with version 1.0.0 and include:

• Consistent variable interfaces

• Comprehensive outputs

• Environment-aware configurations

• Proper resource tagging

Cleanup

To remove all resources:

./scripts/deploy.sh dev destroy
./scripts/deploy.sh staging destroy
./scripts/deploy.sh production destroy

In this blog post, we'll walk through creating a Terraform module for a common infrastructure component - an Application Load Balancer (ALB) with an EC2 instance. This implementation follows the concepts from Chapter 4 (pages 115-139) focusing on "Module Basics", "Inputs", and "Outputs".

Why Use Terraform Modules?

Terraform modules allow you to encapsulate related resources into reusable components. They provide several benefits:

1. Code Reusability: Write once, use many times

2. Abstraction: Hide complexity behind simple interfaces

3. Consistency: Ensure similar infrastructure follows the same patterns

4. Collaboration: Teams can share and use standardized components

Our Infrastructure Components

We'll create a module that deploys:

• An Application Load Balancer (ALB)

• A target group for the ALB

• An EC2 instance running a web server

• Necessary security groups

Module Structure

Here's how we'll structure our module:

modules/
├── alb/
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
├── ec2/
│   ├── main.tf
│   ├── variables.tf
│   └── outputs.tf
└── security_group/
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

The ALB Module

Let's examine the core components of our ALB module:

Inputs (variables.tf)

variable "security_group_id" {
  description = "Security group ID for the ALB"
  type        = string
}

variable "instance_id" {
  description = "Instance ID to attach to the target group"
  type        = string
}

Main Configuration (main.tf)

data "aws_vpc" "default" {
  default = true
}

data "aws_subnets" "default" {
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.default.id]
  }
}

resource "aws_lb" "this" {
  name               = "web-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [var.security_group_id]
  subnets            = data.aws_subnets.default.ids

  tags = {
    Name = "WebALB"
  }
}

resource "aws_lb_target_group" "this" {
  name     = "web-tg"
  port     = 80
  protocol = "HTTP"
  vpc_id   = data.aws_vpc.default.id

  health_check {
    path                = "/"
    protocol            = "HTTP"
    matcher             = "200"
    interval            = 30
    timeout             = 5
    healthy_threshold   = 2
    unhealthy_threshold = 2
  }

  tags = {
    Name = "WebTargetGroup"
  }
}

resource "aws_lb_listener" "this" {
  load_balancer_arn = aws_lb.this.arn
  port              = 80
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.this.arn
  }
}

resource "aws_lb_target_group_attachment" "this" {
  target_group_arn = aws_lb_target_group.this.arn
  target_id        = var.instance_id
  port             = 80
}

Outputs (outputs.tf)

output "alb_dns_name" {
  description = "DNS name of the ALB"
  value       = aws_lb.this.dns_name
}

The EC2 Module

Our EC2 module complements the ALB:

Inputs (variables.tf)

variable "ami_id" {
  description = "AMI ID for the EC2 instance"
  type        = string
}

variable "instance_type" {
  description = "Instance type"
  type        = string
  default     = "t2.micro"
}

variable "security_group_id" {
  description = "Security group ID for the EC2 instance"
  type        = string
}

Main Configuration (main.tf)

resource "aws_instance" "this" {
  ami                    = var.ami_id
  instance_type          = var.instance_type
  vpc_security_group_ids = [var.security_group_id]

  user_data = <<-EOF
              #!/bin/bash
              yum update -y
              yum install -y httpd
              systemctl start httpd
              systemctl enable httpd
              echo "<h1>Hello World from Terraform 30 Day Challenge: Day 8</h1>" > /var/www/html/index.html
              EOF

  tags = {
    Name = "WebServer"
  }
}

Outputs (outputs.tf)

output "instance_id" {
  description = "ID of the EC2 instance"
  value       = aws_instance.this.id
}

output "public_ip" {
  description = "Public IP of the EC2 instance"
  value       = aws_instance.this.public_ip
}

output "public_dns" {
  description = "Public DNS of the EC2 instance"
  value       = aws_instance.this.public_dns
}

The Security Group Module

resource "aws_security_group" "this" {
  name        = "web-server-sg"
  description = "Allow web traffic and SSH access"

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-server-security-group"
  }
}

output "security_group_id" {
  description = "ID of the security group"
  value       = aws_security_group.this.id
}

Root Module Implementation

Now let's see how we use these modules together in our root configuration:

provider "aws" {
  region = var.aws_region
}

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

module "security_group" {
  source = "./modules/security_group"
}

module "ec2" {
  source            = "./modules/ec2"
  ami_id            = data.aws_ami.amazon_linux.id
  instance_type     = var.instance_type
  security_group_id = module.security_group.security_group_id
}

module "alb" {
  source            = "./modules/alb"
  security_group_id = module.security_group.security_group_id
  instance_id       = module.ec2.instance_id
}

output "public_ip" {
  value = module.ec2.public_ip
}

output "public_dns" {
  value = module.ec2.public_dns
}

output "alb_dns_name" {
  value = module.alb.alb_dns_name
}

Key Takeaways

1. Module Composition: We've created three modules that work together to form a complete solution.

2. Input/Output Design: Each module exposes carefully designed inputs and outputs that control its behavior and expose important information.

3. Data Sources: We use data sources to look up information like the default VPC and latest Amazon Linux AMI.

4. Dependencies: Modules can depend on each other through their inputs and outputs, creating an implicit dependency graph.

5. User Data: The EC2 instance is automatically configured with a simple web server through user data.

Next Steps

To improve this implementation, you might consider:

1. Adding variables for all configurable parameters

2. Implementing conditional logic for different environments

3. Adding lifecycle management configurations

4. Incorporating more advanced health check configurations

5. Adding logging and monitoring capabilities

This module provides a solid foundation for deploying ALBs with EC2 instances in AWS, following Terraform best practices for module design and composition.

Introduction

Managing infrastructure as code (IaC) with Terraform requires careful handling of state files, especially in team environments. State isolation and locking are critical to prevent conflicts and ensure smooth collaboration. In this blog, we'll explore:

• State File Isolation (Workspaces & File Layouts)

• Remote State Storage (S3 Backend)

• State Locking (DynamoDB)

1. State File Isolation

State isolation ensures that changes in one environment (e.g., dev) don’t accidentally affect another (e.g., prod).

Option 1: Isolation via Workspaces

Terraform workspaces allow multiple state files within the same configuration.

Commands:

# Create workspaces for dev, staging, prod
terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

# Switch between workspaces
terraform workspace select dev

Pros: Quick setup, no code duplication.
Cons: Shared modules may lead to accidental cross-environment changes.

Option 2: Isolation via File Layouts

A more explicit approach is using separate directories per environment.

Directory Structure:

terraform/
├── dev/
│   ├── main.tf
│   └── backend.tf  
├── staging/
│   ├── main.tf
│   └── backend.tf  
└── prod/
    ├── main.tf
    └── backend.tf

Pros: Complete isolation, fewer risks of overlap.
Cons: More files to maintain.

2. Remote State Storage (S3 Backend)

Storing state remotely in Amazon S3 ensures:
Team accessibility
Versioning & Encryption
State locking

Create an S3 Bucket & DynamoDB Table

aws s3api create-bucket \
    --bucket simi-ops-terraform-state \ 
    --region us-west-2 --create-bucket-configuration \
    LocationConstraint=us-west-2

aws dynamodb create-table \
    --table-name simi-ops-terraform-locks \
    --attribute-definitions AttributeName=LockID,AttributeType=S \
    --key-schema AttributeName=LockID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
    --region us-west-2

Configure Terraform Backend

terraform {
  backend "s3" {
    bucket  = "simi-ops-terraform-state"
    key     = "web-server/terraform.tfstate"
    region  = "us-west-2"
    dynamodb_table = "simi-ops-terraform-locks"
    encrypt = true
  }
}

3. State Locking with DynamoDB

Prevents concurrent state modifications by locking the state file during operations.

How It Works:

1. User A runs terraform apply → DynamoDB creates a lock entry.

2. User B tries to modify infrastructure → Terraform checks the lock and blocks changes until User A finishes.

3. Once User A completes, the lock is released.

Testing State Locking

• User 1: Runs terraform apply → acquires lock.

• User 2: Attempts terraform apply → gets an error:

    Error: Error acquiring the state lock
    Lock Info: ID: <lock-id> | Status: Locked
  

Prevents corruption from overlapping changes.

Conclusion

By implementing:
Workspaces / File Layouts → Isolate environments.
S3 Backend → Securely store state.
DynamoDB Locking → Prevent conflicts.

Introduction to Terraform State

When working with Terraform, one of the most critical concepts to understand is Terraform state. As I recently worked through Chapter 3 (pages 81-113) of my Terraform studies, I gained valuable insights into what Terraform state is, why it's important, and how to manage it effectively across teams.

What is Terraform State?

Terraform state is stored in a file named terraform.tfstate by default. This JSON-formatted file serves several essential purposes:

1. Mapping to Real World: It keeps track of the resources Terraform manages and their current settings

2. Metadata Storage: Stores dependencies between resources that aren't apparent from your configuration

3. Performance: For large infrastructures, it helps Terraform run operations more efficiently

4. Sync Mechanism: Enables teams to work together by knowing the current infrastructure state

The state file contains sensitive information (like database passwords in plaintext), so it should always be protected.

Shared Storage for State Files

Working with state files locally is fine for individual projects, but when collaborating with teams, we need a better solution. The chapter highlighted several problems with local state files:

• Team Conflicts: Multiple team members can't easily work together

• Data Loss Risk: Local files can be accidentally deleted

• Security Issues: Sensitive data isn't properly protected

The solution? Remote state storage - storing the state file in a shared, secure location that the whole team can access.

Managing State Across Teams

For team collaboration, we need to consider:

1. Locking Mechanisms: Prevent multiple simultaneous operations that could corrupt state

2. Access Controls: Ensure only authorized personnel can modify infrastructure

3. Versioning: Track changes to infrastructure state over time

4. Audit Logs: Monitor who made what changes and when

Hands-on Activity: Deploying Infrastructure and Configuring Remote State

Part 1: Deploying Infrastructure and Inspecting State

From the previous Terraform code, I used it to create the S3 bucket to store the state file

After running terraform apply, I inspected the generated terraform.tfstate file. The JSON structure showed all the resource attributes and metadata. Key observations:

• Each resource has a unique address

• The file tracks the actual state of cloud resources

• Sensitive data is stored in plaintext (highlighting the need for secure storage)

Part 2: Configuring Remote State with AWS S3

To implement remote state storage, I followed these steps:

Created an S3 bucket used CLI

aws s3api create-bucket --bucket simi-ops-terraform-state \  
    --region us-west-2 --create-bucket-configuration \ 
    LocationConstraint=us-west-2

Configured Terraform to use the S3 backend:

terraform {
  backend "s3" {
    bucket  = "simi-ops-terraform-state"
    key     = "web-server/terraform.tfstate"
    region  = "us-west-2"
    encrypt = true
  }
}

After initializing with terraform init, Terraform automatically migrated my local state to S3. Now my state is:

• Stored securely in an encrypted S3 bucket

Key Takeaways

1. State is essential: Terraform relies on it to map your configuration to real resources

2. Local state isn't for teams: Remote storage solves collaboration challenges

3. AWS S3 is a robust solution: Especially when combined with DynamoDB for locking

4. Security matters: Always encrypt state files and control access carefully

Next Steps

• Implement DynamoDB to store lock file

• Implement Versioning

• Implement IAM policies for fine-grained state access control

• Explore Terraform Cloud for enhanced collaboration features

• Set up state file auditing and change notifications

Understanding Terraform state management has been a game changer for my infrastructure-as-code workflows. The shift from local to remote state might seem like extra work initially, but the benefits for team collaboration and security make it absolutely worth the effort.

Deploying a Load Balancer

A Load Balancer distributes incoming traffic across multiple servers to improve availability and fault tolerance. In our Terraform configuration, we deploy an Application Load Balancer - ALB, to manage traffic to our web servers.

Why Use a Load Balancer?

• High Availability: If one instance fails, traffic routes to healthy instances.

• Scalability: Easily add more instances behind the ALB.

• Efficient Traffic Distribution: Balances load across multiple servers.

Activity: Scaling Web Servers & Managing State

Step 1: Modify Configuration for Multiple Instances

resource "aws_instance" "web_server" {
  count         = 3  # Creates 3 instances
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.instance_type
  vpc_security_group_ids = [aws_security_group.ec2_sg.id]
  user_data     = file("user_data.sh")
  tags = {
    Name = "WebServer-${count.index}"
  }
}

Step 2: Update ALB Target Group Attachments

resource "aws_lb_target_group_attachment" "web_attachment" {
  count            = length(aws_instance.web_server)
  target_group_arn = aws_lb_target_group.web_tg.arn
  target_id        = aws_instance.web_server[count.index].id
  port             = 80
}

Step 3: Initialize & Apply

terraform init
terraform plan
terraform apply

Step 4: Verify Scaling

• Check the AWS EC2 Console to see multiple instances.

• Access the ALB DNS (output "alb_dns_name") to confirm load balancing works.

Conclusion

• Load Balancers improve scalability and availability.

• Terraform State is crucial for tracking infrastructure.

• Remote State Storage (S3, Terraform Cloud) enables team collaboration.

• Avoid Manual State Edits to prevent corruption.

What I Learned

1. Using Variables for Flexibility

Variables in Terraform allow us to parameterize our configurations, making them more adaptable. Instead of hardcoding values like region or instance_type, we can define them as variables and pass different values as needed.

In my implementation, I defined two variables in variables.tf:

variable "aws_region" {
  description = "AWS region to deploy resources"
  type        = string
  default     = "us-west-2"
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t2.micro"
}

These variables were then referenced in the main configuration:

provider "aws" {
  region = var.aws_region
}

resource "aws_instance" "web_server" {
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.instance_type
  # ... rest of the config
}

This makes it easy to change regions or instance types without modifying the core infrastructure code.

2. Leveraging Data Sources

Data sources allow Terraform to fetch external information (like the latest Amazon Linux AMI) and use it in configurations. Instead of hardcoding an AMI ID (which changes frequently), I used:

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

This dynamically retrieves the latest Amazon Linux 2 AMI, ensuring my EC2 instance always uses an up to date image.

3. Security Group & User Data Configuration

I also defined a security group to allow SSH (22), HTTP (80), and HTTPS (443) traffic:

resource "aws_security_group" "ec2_sg" {
  name        = "web-server-sg"
  description = "Allow web traffic and SSH access"

  # Ingress rules for SSH, HTTP, HTTPS
  # Egress rule allowing all outbound traffic
}

Additionally, I used user_data to automatically install and configure an Apache web server upon instance launch:

user_data = <<-EOF
            #!/bin/bash
            yum update -y
            yum install -y httpd
            systemctl start httpd
            systemctl enable httpd
            echo "<h1>Hello World from Terraform 30 Day Challenge: Day 4</h1>" > /var/www/html/index.html
            EOF

Results

After running terraform apply, my AWS infrastructure was successfully deployed:

• A t2.micro EC2 instance running Amazon Linux 2

• A security group allowing web and SSH access

• An Apache web server serving a "Hello World" page

Key Takeaways

• Variables make Terraform configurations reusable and flexible.

• Data sources help fetch dynamic external data (like AMIs).

• Security groups define inbound/outbound traffic rules.

• User data automates instance initialization.

Looking forward to Day 5, where I’ll explore more advanced Terraform concepts

I chose AWS as my cloud provider since it's widely used and integrates well with Terraform. Below, I’ll walk through the steps I took to complete this task.

Task 1: Designing the Architecture Diagram

Since I used AWS, I designed a simple architecture in draw.io showing:

Single Server Deployment

• Region: us-west-2

• Instance Type: t2-micro

  

Web Server Deployment

• Region: us-west-2

• Instance Type: t2-micro

  

Key Components:

1. EC2 Instance – Hosts the Apache web server.

2. Security Group – Controls inbound/outbound traffic.

3. User Data Script – Automates web server setup

Task 2: Writing Terraform Code for a Basic Web Server

Step 1: Setting Up Terraform

Before writing any code, I ensured:

• Terraform was installed (terraform --version).

• AWS CLI was configured with my credentials (aws configure).

Step 2: Writing the Terraform Configuration

I created a new directory for this project and wrote a main.tf file with the following code:

The terraform code for basic single server:

# Configure the AWS Provider
provider "aws" {
  region = "us-west-2" 
}

resource "aws_security_group" "ec2_sg" {
  name        = "ec2-security-group"
  description = "Allow SSH inbound traffic"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] 
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "my_single_ec2_server" {
  ami           = "ami-04999cd8f2624f834" 
  instance_type = "t2.micro"              

  vpc_security_group_ids = [aws_security_group.ec2_sg.id]

  tags = {
    Name = "Single-Server"
  }
}


output "instance_public_ip" {
  description = "Public IP address of the EC2 instance"
  value       = aws_instance.my_single_ec2_server.public_ip
}

Terraform code for Basic Web Server:

# Configure the AWS Provider
provider "aws" {
  region = "us-west-2"
}
resource "aws_security_group" "ec2_sg" {
  name        = "web-server-sg"
  description = "Allow web traffic and SSH access"

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] 
  }

  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-server-security-group"
  }
}

resource "aws_instance" "web_server" {
  ami           = "ami-04999cd8f2624f834" 
  instance_type = "t2.micro"             
  vpc_security_group_ids = [aws_security_group.ec2_sg.id]

  user_data = <<-EOF
              #!/bin/bash
              yum update -y
              yum install -y httpd
              systemctl start httpd
              systemctl enable httpd
              echo "<h1>Hello World from Terraform 30 Day Challenge</h1>" > /var/www/html/index.html
              EOF

  tags = {
    Name = "WebServer"
  }
}

output "public_ip" {
  description = "Public IP address of the web server"
  value       = aws_instance.web_server.public_ip
}

output "public_dns" {
  description = "Public DNS name of the web server"
  value       = aws_instance.web_server.public_dns
}

Challenges Faced & Solutions

1. AMI ID Variability – Had to ensure I used the correct Amazon Linux 2 AMI for us-west-2

   • Solution: Checked the AWS AMI catalog.

2. Security Group Misconfiguration – Initially blocked HTTP traffic.

   • Solution: Verified ingress rules for port 80.

Activity: Setting up AWS and Terraform Development Environment

Set up your AWS account.

• Go to AWS website and click "Create an AWS Account"

• Complete the registration process providing email, password, and account information

• Enter payment information, credit card required, even for free tier

• Complete phone verification

• Select a support plan, Free tier recommended for beginners

Install Terraform locally.

• Download Terraform from terraform.io/downloads

• Extract the downloaded zip file

• Add Terraform to your system PATH:

  • Windows: Move terraform.exe to a directory in your PATH or add its location to PATH

  • macOS/Linux: Move the terraform binary to /usr/local/bin/

• Verify installation by opening a terminal and typing: terraform -version

Install AWS CLI and configure it.

• Download AWS CLI:

  • Windows: Download and run the MSI installer

  • macOS: Run brew install awscli or download the PKG installer

  • Linux: Run appropriate package manager command (e.g., apt install awscli)

• Configure AWS CLI by running: aws configure

• Enter your AWS Access Key ID, Secret Access Key, default region, and output format

• Find your access keys in the AWS Console under IAM → Security credentials

  

Install Visual Studio Code (VSCode) and add the AWS plugin.

• Download VSCode from code.visualstudio.com

• Install VSCode on your system

• Open VSCode and navigate to Extensions (Ctrl+Shift+X or Cmd+Shift+X)

• Search for "AWS Toolkit" and install it

• Also install the "HashiCorp Terraform" extension for Terraform support

  

Configure your VSCode to work with AWS.

• Click on the AWS icon in the activity bar

• Select "Connect to AWS"

• Choose to use the credentials from your AWS CLI configuration

• Verify connection by expanding your region in the AWS Explorer panel

• Configure Terraform settings in VSCode:

  • Go to Settings (Ctrl+,)

  • Search for "terraform" and adjust formatting settings as needed

  • Enable auto-formatting and validation features

IaC is a way to automate infrastructure in your environments, mostly cloud based, but it can also be used on prem. Essentially, it means managing and provisioning your infrastructure through code instead of manual processes. This brings a ton of benefits, like consistency, scalability, and speed. The beauty of it lies in its ability to bring software development practices like version control, testing, and CI/CD to your infrastructure.

Tools like Terraform, CloudFormation, and Ansible are at the forefront of this movement, allowing engineers to treat their infrastructure like any other codebase, leading to fewer errors and faster deployments.

My focus with would specifically is to deeply understand its declarative language and how it interacts with various cloud providers to provision and manage resources efficiently. I'm excited to dive deeper into modules, state management, and best practices for collaborative IaC development. Over the next 30 days, I aim to achieve a solid foundational understanding of the core concepts.

I want to successfully deploy and manage a complete application stack using Terraform, covering networking, compute, and database services. Furthermore, I hope to gain practical experience with state file management, understand remote backends, and explore strategies for handling sensitive data securely. See you on day 30

Day 1 tasks

Completed the assigned tasks for the day:

1. Reading: Chapter 1 of "Terraform: Up & Running"

2. Complete a hands on lab

3. Blog Post: "What is IAC and its benefits ,."

4. Social Media Post: "💻 Just installed Terraform, AWS CLI, and configured my AWS environment with VSCode. Ready to deploy some infrastructure! #TerraformSetup #AWS #DevOps"

Hands on Lab

This lab setting up a VPC through a two ways. Initially, we manually configured a test VPC in the AWS console, encompassing the creation of subnets, route tables, an Elastic IP, and both Internet and NAT Gateways. This approach built a foundational understanding of each component's role.

Following this, the we transitioned to Terraform, automating the exact same VPC setup. This phase emphasized the principles of Infrastructure as Code, demonstrating how defining it from the sample code provided.

Manual VPC Console Setup

Then Set up Terraform to do the same

Using the code examples my main.tf

# Configure the AWS Provider
provider "aws" {
  region = "us-west-2"
}

#Retrieve the list of AZs in the current AWS region
data "aws_availability_zones" "available" {}
data "aws_region" "current" {}

#Define the VPC
resource "aws_vpc" "vpc" {
  cidr_block = var.vpc_cidr

  tags = {
    Name        = var.vpc_name
    Environment = "demo_environment"
    Terraform   = "true"
  }
}

#Deploy the private subnets
resource "aws_subnet" "private_subnets" {
  for_each          = var.private_subnets
  vpc_id            = aws_vpc.vpc.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, each.value)
  availability_zone = tolist(data.aws_availability_zones.available.names)[each.value]

  tags = {
    Name      = each.key
    Terraform = "true"
  }
}

#Deploy the public subnets
resource "aws_subnet" "public_subnets" {
  for_each                = var.public_subnets
  vpc_id                  = aws_vpc.vpc.id
  cidr_block              = cidrsubnet(var.vpc_cidr, 8, each.value + 100)
  availability_zone       = tolist(data.aws_availability_zones.available.names)[each.value]
  map_public_ip_on_launch = true

  tags = {
    Name      = each.key
    Terraform = "true"
  }
}

#Create route tables for public and private subnets
resource "aws_route_table" "public_route_table" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block     = "0.0.0.0/0"
    gateway_id     = aws_internet_gateway.internet_gateway.id
    #nat_gateway_id = aws_nat_gateway.nat_gateway.id
  }
  tags = {
    Name      = "demo_public_rtb"
    Terraform = "true"
  }
}

resource "aws_route_table" "private_route_table" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block     = "0.0.0.0/0"
    # gateway_id     = aws_internet_gateway.internet_gateway.id
    nat_gateway_id = aws_nat_gateway.nat_gateway.id
  }
  tags = {
    Name      = "demo_private_rtb"
    Terraform = "true"
  }
}

#Create route table associations
resource "aws_route_table_association" "public" {
  depends_on     = [aws_subnet.public_subnets]
  route_table_id = aws_route_table.public_route_table.id
  for_each       = aws_subnet.public_subnets
  subnet_id      = each.value.id
}

resource "aws_route_table_association" "private" {
  depends_on     = [aws_subnet.private_subnets]
  route_table_id = aws_route_table.private_route_table.id
  for_each       = aws_subnet.private_subnets
  subnet_id      = each.value.id
}

#Create Internet Gateway
resource "aws_internet_gateway" "internet_gateway" {
  vpc_id = aws_vpc.vpc.id
  tags = {
    Name = "demo_igw"
  }
}

#Create EIP for NAT Gateway
resource "aws_eip" "nat_gateway_eip" {
  domain     = "vpc"
  depends_on = [aws_internet_gateway.internet_gateway]
  tags = {
    Name = "demo_igw_eip"
  }
}

#Create NAT Gateway
resource "aws_nat_gateway" "nat_gateway" {
  depends_on    = [aws_subnet.public_subnets]
  allocation_id = aws_eip.nat_gateway_eip.id
  subnet_id     = aws_subnet.public_subnets["public_subnet_1"].id
  tags = {
    Name = "demo_nat_gateway"
  }
}

variables.tf

variable "aws_region" {
  type    = string
  default = "us-west-2"
}

variable "vpc_name" {
  type    = string
  default = "demo_vpc"
}

variable "vpc_cidr" {
  type    = string
  default = "10.0.0.0/16"
}

variable "private_subnets" {
  default = {
    "private_subnet_1" = 1
    "private_subnet_2" = 2
    "private_subnet_3" = 3
  }
}

variable "public_subnets" {
  default = {
    "public_subnet_1" = 1
    "public_subnet_2" = 2
    "public_subnet_3" = 3
  }
}

And terraform install, and terraform init

Did a terraform plan

And terraform apply

And VPC resources created using terraform

And lastly using terraform destroy, to automatically remove the created resources

At the end of day 1 I completed the learning cycle and understood the full lifecycle management, I then used terraform destroy to automatically remove all the created resources, demonstrating Terraform's capability for efficient cleanup and resource deprovisioning. This experience with the entire plan -> apply -> destroy cycle has been invaluable in solidifying my understanding.

Your Cloud Detective,AWS GuardDuty

Think of AWS GuardDuty as a detective that’s always on the lookout for suspicious activity. It’s a threat detection service that continuously monitors your AWS environment for signs of trouble. it uses machine learning and analyzes data from various sources, like AWS CloudTrail logs, VPC Flow Logs, and DNS logs, to spot unusual behavior.

For example, if someone tries to log in to your account from a strange location or if an EC2 instance starts communicating with a known malicious IP address, it will flag it. It’s like having a security guard who’s always watching and ready to raise the alarm.

If you want to detect potential threats in real time, like unauthorized access, compromised instances, or suspicious network activity, GuardDuty is your tool.

The Vulnerability Scanner, AWS Inspector

AWS Inspector is designed to find vulnerabilities in your applications and infrastructure. It automatically assesses your resources, such as EC2 instances, and checks for common security issues, like open ports, missing patches, or misconfigurations.

By running automated security assessments, it provides a detailed report with recommendations on how to fix the issues it finds. It’s not a real time like GuardDuty but a more of a periodic check-up to make sure everything is secure.

If you’re looking to identify and fix vulnerabilities in your applications or infrastructure, Inspector is the right choice. It’s especially useful before deploying new applications or after making significant changes to your environment. Think of it as a way to ensure your systems are secure before they go live.

Your DDoS Bodyguard, AWS Shield

AWS Shield is all about protecting your applications from Distributed Denial of Service (DDoS) attacks. These attacks can overwhelm your systems with traffic, making them unavailable to legitimate users. Shield comes in two versions: Standard and Advanced.

• Shield Standard is automatically included with all AWS accounts and provides basic protection against common DDoS attacks.

• Shield Advanced is a paid service that offers enhanced protection, including 24/7 access to the AWS DDoS Response Team, detailed attack reports, and financial protection against scaling costs during an attack.

If you’re running applications that need to be highly available and you’re concerned about DDoS attacks, Shield is a must. Shield Advanced is ideal for businesses that need extra protection and support, especially if they’re running critical workloads.

How They Work Together

While they all serve different purposes, they can work together to provide a comprehensive security strategy. Here’s how:

• GuardDuty monitors for threats in real time, helping you detect and respond to suspicious activity.

• Inspector identifies vulnerabilities in your applications and infrastructure, giving you a chance to fix them before they’re exploited.

• Shield protects your applications from DDoS attacks, ensuring they stay online and available.

For example, Inspector might find an open port on one of your EC2 instances. You close the port, but GuardDuty later detects unusual traffic from that instance, indicating a potential compromise. Meanwhile, Shield is protecting your application from being taken offline by a DDoS attack. Together, these tools create a layered defense that keeps your AWS environment secure.