What makes the Punk Security CTF tick?

Our DevSecOps CTF is May 4th this year, but what makes it tick?

Punk Security was founded on the principle of giving back to the community. We talk at conferences globally and build and maintain open-source tools (dnsReaper, SMBeagle, pwnSpoof, and Secret Magpie). We’ve always wanted to do a DevSecOps CTF and our growth has meant that we can now make that a reality!

Our birthday CTF runs on AWS and has been designed from the very start to be scalable and secure. We’re inviting hackers from all around the globe onto our AWS platform, so our security has to be unquestionably tight!

Defining the user experience

From the start we had a clear vision for our CTF to make sure:

It was SUPER EASY for people to get playing
We could use REAL WORLD technology
Players were not held back by being on windows, or linux, or mac

We’ve played a LOT of CTFs in our time and they typically end up with the player installing all sorts of things on their laptops to complete challenges. Weird steganography tools, python packages and even attack scripts. We didn’t want that.

What we decided on was using our own expertise to build challenges within containers on AWS so that every player got an identical, but completely isolated experience. With this model, we could give a player access to their own Kubernetes cluster or Jenkins controller. No emulation or simulation, the real deal.

Our challenges would require either SSH access to a linux terminal, or web access to a browser-based application (like Jenkins) so we decided to use Apache Guacamole. With this technology bolted on, players can access Linux terminal straight from their web browsers.

Guacamole

So, let’s choose some technology…

The CTF framework

We chose early on to use CTFd as the framework for our CTF platform. CTFd is one of the biggest open-source CTF platforms, with 4.5k stars over on Github and 1.7k forks, so it’s a great candidate. It’s written in Python and has some great extensibility through a plugin architecture which would allow us to bolt in the AWS automation we wanted.

NOTE: At this point it’s worth a quick mention of RootTheBox too. It’s another fantastic CTF framework with a really strong community. It uses some slightly different mechanisms to add a better “story” to your CTF but ultimately didn’t feel right for us.

Authentication

We never really wanted to handle authentication. It’s messy and credentials are a valuable target, so are best avoided altogether.

There are a number of authentication providers we could leverage, like Facebook or Github auth, that make life easier. What we actually went for was ctftime.org which is a service dedicated to organising and promoting CTFs.

CTFtime supports a few authentication providers themselves, like Github, and adds some extra benefits. Our scores automatically synchronise back to CTFtime so teams can go on to compete in other CTFs and see their score over time.

Apache Guacamole also requires authentication, so we wrote a custom integration that allows each challenge to be uniquely authenticated via an encrypted JWT token. This also meant that we could scale Apache Guacamole horizontally without having to worry about tracking the state in a database!

Ephemeral challenges on AWS

So how to tackle creating safe and scalable challenges on AWS?

Every challenge consists of one or more containers, which are bundled together in an ECS task definition and stored in one of our private Github repositories. We have a few terraform modules which MASSIVELY reduces the time to improve, iterate and update our challenges. If we need to replace some ASCII art, we can change it in a parent container image and have terraform rebuild every challenge in 10 minutes or so.

To prevent rebuilds we inject flags, passwords and SSH keys at runtime by modifying the ECS task definitions when we launch a challenge. This keeps our launch times generally quite low, around 30 seconds or so on average.

For security, we completely isolate the CTF control plane and player challenge environments by deploying them to separate VPCs within AWS. Apache Guacamole JWT authentication means our control plane never actually talks to Apache Guacamole, we can safely trust the players’ browser to handle that for us. In short, we are secure and scalable but more importantly, we can build awesome challenges that really let players break things!

Overall architecture

So…

Players access the CTF over at https://ctf.punksecurity.co.uk, which then sends them to CTFtime to set up an account, create a team and sign up for our CTF. Once logged in, the player comes back to us to see all the challenges they can play.

Whilst the player just sees our CTF control panel, behind the scenes they could be on any of our highly available CTF containers.

The player picks an awesome challenge and selects it. Behind the scenes, automation kicks in to create an ECS task for their challenge. Typically this takes around 30 seconds and uses AWS Fargate so that it can scale as much as it needs to. These challenges get a unique, per-player flag injected into them so flags can’t just be passed around between teams.

On occasion, we cannot use AWS Fargate. If you play one of our Kubernetes challenges then you’re actually using our dedicated EC2 server pool. Whilst this isn’t as scalable as AWS Fargate, it’s allowed us to shave a few seconds off the challenge launch time.

Once the challenge is ready and passing health checks (yup, all of our challenges have health checks) the player gets a connect button. This issues the player with a unique authentication token and sends them over to one of our Apache Guacamole web servers to access their challenge in the browser. We can scale our number of Guacamole instances as required, but as standard, we are resilient across AWS datacenters.

That’s a lot to take in, so it’s probably best to see the diagram…

Architecture

Introducing Cloudflare

So those hacker types love a good Denial of Service, so we decided to put our entire platform behind Cloudflare which we have found to be fantastic at soaking up DDoS attacks.

With some clever headers and IP filters, those pesky attacker types cannot circumvent our protections.

Building challenges

Our challenges have a lot of moving parts and leverage leading tools and approaches to deliver a really awesome game experience.

We’ve used VNC and Google Chrome to provide players with a fully working, but restricted, browser within their browser. Anything you do in the game won’t show up in your history and you will never be able to download files (or viruses) from inside the game to your machine.

We use cloud-native services, like AWS Route53, to generate actual real-world attacks like subdomain takeovers.

We use selenium to control a containerised Chrome browser and simulate real users of our web apps, which allows us to write real XSS scenarios for you to attack!

… And of course, we use our own cutting-edge tools, generating attack logs with pwnSpoof and real-world scenarios where the player can find a misplaced secret and abuse it to devastating effect.

So good we built it twice!

You should never develop on a production system… even if that system is a novel CTF to celebrate your birthday.

Our entire CTF platform is deployed as Infrastructure as Code with terraform and docker. Everything from the challenges, to the AWS environments and cloudflare configuration.

Not only does this give us change control, peer review and an audit log… It also allows us to maintain a development and production instance of the entire platform. We can keep a rapid pace of development, whilst our production instance receives only stable and tested updates.