From be2aeeccdfff97e03c3276081ee5788f2b0be32d Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Fri, 24 Sep 2021 15:43:22 -0600 Subject: [PATCH 01/10] initial work on new getting started tutorial Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 140 +++++++++------------------------ 1 file changed, 36 insertions(+), 104 deletions(-) diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index d5548f98..d29efe58 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -2,123 +2,55 @@ slug: /1003/get-started/ --- -# Get started with Dagger +# Get Started with Dagger -In this guide, you will learn the basics of Dagger by interacting with a pre-configured environment. -Then you will move on to creating your environment from scratch. - -Our pre-configured environment deploys a simple [React](https://reactjs.org/) -application to a unique hosting environment created and managed by us, the Dagger team, for this tutorial. -This will allow you to deploy something "real" right away without configuring your infrastructure first. - -In later guides, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, +In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple +project deploys a [React](https://reactjs.org/) application to your local machine via docker. In later tutorials, +you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, how to share access to your infrastructure in the same way that we share access to ours now. -## Initial setup +This tutorial does involve writing CUE, so if you haven’t already, be sure to read [What is CUE?](../introduction/1005-what_is_cue.md) + +In this tutorial we will learn: + +- How to initialize and structure a Dagger project +- About Dagger concepts such as + - the plan + - environments + - inputs and outputs +- How to write CUE for Dagger +- How to deploy an application using `dagger up` + +## Deploy an Application Locally + +The following instructions assume you are working locally, but could just as easily be run on a remote +machine into which you have a shell. ### Install Dagger -First, make sure [you have installed Dagger on your local machine](../1001-install.md). +First, make sure [you have installed Dagger](../1001-install.md). You can run `dagger version` to ensure +you have the latest installed and working. For the sake of brevity and simplicity we will create directories under +your home directory, but feel free to replace `~/` with a path that works best for you. -### Setup example app +### Create a Dagger Project -You will need a local copy of the [Dagger examples repository](https://github.com/dagger/examples). -NOTE: you may use the same local copy across all tutorials. +First we need a directory that will contain our `.cue` files and a `.dagger` directory which stores metadata about environments. First, create a new directory for our todoapp, then initialize the project: -```shell -git clone https://github.com/dagger/examples +```bash +mkdir ~/todoapp +cd ~/todoapp +dagger init ``` -Make sure that all commands are run from the `todoapp` directory: +If you now run `ls -la` you will see 2 new directories: -```shell -cd examples/todoapp -``` +- The `.dagger` directory will store metadata about _environments_, _inputs_, and _outputs_ which we will cover shortly. +- The `cue.mod` directory stores libraries such as [dagger/universe](https://github.com/dagger/universe) which can be _imported_ into your Dagger _plan_. -### Import the tutorial key +Dagger will load all `.cue` files recursively in the current Dagger project. More directories can be added to help organize code. -Dagger natively supports encrypted secrets: when a user inputs a value marked as secret -(for example, a password, API token, or ssh key) it is automatically encrypted with that user's key, -and no other user can access that value unless they are explicitly given access. +> Note that Dagger, like the CUE CLI command, will only load CUE files from the `cue.mod` directory in response to `import` statements. -In the interest of security, Dagger has no way _not_ to encrypt a secret value. -But this causes a dilemma for this tutorial: how do we give unrestricted, public access to our -(carefully sandboxed) infrastructure so that anyone can deploy to it? +### Write a Dagger Plan -To solve this dilemma, we included the private key used to encrypt the tutorial's secret inputs. -Import the key to your Dagger installation, and you're good to go: - -```shell -./import-tutorial-key.sh -``` - -## First deployment - -Now that your environment is set up, you are ready to deploy: - -```shell -dagger up -``` - -That's it! You have just made your first deployment with Dagger. - -The URL of your newly deployed app should be visible towards the end of the command output. -If you visit that URL, you should see your application live! - -## Code, deploy, repeat - -This environment is pre-configured to deploy from the `./todoapp` directory, -so you can make any change you want to that directory, then deploy it with `dagger up`. -You can even replace our example React code with any React application! - -NOTE: you don't have to commit your changes to the git repository before deploying them. - -## Under the hood - -This example showed you how to deploy and develop an application that is already configured with Dagger. Now, let's learn a few concepts to help you understand how this was put together. - -### The Environment - -An Environment holds the entire deployment configuration. - -You can list existing environment from the `./todoapp` directory: - -```shell -dagger list -``` - -You should see an environment named `s3`. You can have many environments within your app. For instance, one for `staging`, one for `dev`, etc... - -Each environment can have a different kind of deployment code. For example, a `dev` environment can deploy locally; a `staging` environment can deploy to a remote infrastructure, and so on. - -### The plan - -The plan is the deployment code that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan: - -```shell -ls -l ./s3 -``` - -Any code change to the plan will be applied during the next `dagger up`. - -### The inputs - -The plan can define one or several `inputs`. Inputs may be configuration values, artifacts, or encrypted secrets provided by the user. Here is how to list the current inputs: - -```shell -dagger input list -``` - -The inputs are persisted inside the `.dagger` directory and pushed to your git repository. That's why this example application worked out of the box. - -### The outputs - -The plan defines one or several `outputs`. They can show helpful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all outputs: - -```shell -dagger output list -``` - -## What's next? - -At this point, you have deployed your first application using Dagger and learned some dagger commands. You are now ready to [learn more about how to program Dagger](./1004-first-env.md). +A Dagger _plan_ is written in CUE and declaratively expresses the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. From 00e10219ca04ebf018a982571562fba3ee446caa Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Mon, 27 Sep 2021 12:33:01 -0600 Subject: [PATCH 02/10] additional progress Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 58 +++++++++++++++++++++++++++++++--- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index d29efe58..cd6be47f 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -24,13 +24,13 @@ In this tutorial we will learn: ## Deploy an Application Locally The following instructions assume you are working locally, but could just as easily be run on a remote -machine into which you have a shell. +machine into which you have a shell. For the sake of brevity and simplicity we will create directories under +your home directory, but feel free to replace `~/` with a path that works best for you. ### Install Dagger First, make sure [you have installed Dagger](../1001-install.md). You can run `dagger version` to ensure -you have the latest installed and working. For the sake of brevity and simplicity we will create directories under -your home directory, but feel free to replace `~/` with a path that works best for you. +you have the latest installed and working. ### Create a Dagger Project @@ -53,4 +53,54 @@ Dagger will load all `.cue` files recursively in the current Dagger project. Mor ### Write a Dagger Plan -A Dagger _plan_ is written in CUE and declaratively expresses the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. +A Dagger _plan_ is written in CUE and expresses the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. Unlike traditional glue code written in an scripting language (e.g.: Bash, PowerShell), a Dagger plan is _declarative_ rather than _imperative_. This frees us from thinking about order of operations, since Dagger will infer dependendencies and calculate correct order on its own. + +First create a directory to hold our plan, separate from our application code: + +```shell +mkdir ./plan +``` + +Next, create a file in `plan/` called `todoapp.cue` with the following content + +```cue +package todoapp + +import ( + "alpha.dagger.io/dagger" + "alpha.dagger.io/dagger/stream" + "alpha.dagger.io/js/yarn" +) + +// Source code of the sample application +source: dagger.#Artifact & dagger.#Input + +// Build the source code using Yarn +app: yarn.#Package & { + "source": source +} + +``` + +### Create an Environment + +```shell +dagger new local -p ./plan +dagger list +``` + +### Define Input Values per Environment + +```shell +dagger input list +``` + +```text +Input Value Set by user Description +app.source dagger.#Artifact false Application source code +``` + +```shell +dagger -e local input dir app.source ./app +``` + From ef66f0bc69b23e71c192600763fea6aa421121b4 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Fri, 1 Oct 2021 15:11:15 -0600 Subject: [PATCH 03/10] new getting started tutorial Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 185 ++++++++++++++++++++++++++------- 1 file changed, 149 insertions(+), 36 deletions(-) diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index cd6be47f..ef5c31ce 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -4,10 +4,7 @@ slug: /1003/get-started/ # Get Started with Dagger -In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple -project deploys a [React](https://reactjs.org/) application to your local machine via docker. In later tutorials, -you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, -how to share access to your infrastructure in the same way that we share access to ours now. +In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple project deploys a [React](https://reactjs.org/) application to your local machine via docker. In later tutorials, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, how to share access to your infrastructure in the same way that we share access to ours now. This tutorial does involve writing CUE, so if you haven’t already, be sure to read [What is CUE?](../introduction/1005-what_is_cue.md) @@ -23,28 +20,43 @@ In this tutorial we will learn: ## Deploy an Application Locally -The following instructions assume you are working locally, but could just as easily be run on a remote -machine into which you have a shell. For the sake of brevity and simplicity we will create directories under -your home directory, but feel free to replace `~/` with a path that works best for you. +The following instructions assume you are working locally, but could just as easily be run on a remote machine into which you have a shell. ### Install Dagger -First, make sure [you have installed Dagger](../1001-install.md). You can run `dagger version` to ensure -you have the latest installed and working. +First, make sure [you have installed Dagger](../1001-install.md). You can run `dagger version` to ensure you have the latest installed and working. ### Create a Dagger Project -First we need a directory that will contain our `.cue` files and a `.dagger` directory which stores metadata about environments. First, create a new directory for our todoapp, then initialize the project: +First clone the [Dagger examples repository](https://github.com/dagger/examples), change directories to the `todoapp/` and list its contents: + +> Note that all tutorials will operate from the todoapp directory. ```bash -mkdir ~/todoapp -cd ~/todoapp -dagger init +git clone https://github.com/dagger/examples.git +cd examples/todoapp +ls -la ``` -If you now run `ls -la` you will see 2 new directories: +This React application will use Yarn to build a static website with the following directories and files. -- The `.dagger` directory will store metadata about _environments_, _inputs_, and _outputs_ which we will cover shortly. +```bash +-rw-r--r-- ... 794 Sep 7 10:09 package.json +drwxr-xr-x ... 256 Sep 7 10:09 public +drwxr-xr-x ... 192 Sep 29 11:17 src +-rw-r--r-- ... 465514 Sep 29 11:17 yarn.lock +``` + +Now we need to initialize this directory as a Dagger _project_ (and relist directories): + +```bash +dagger init +ls -la +``` + +You will now see 2 new directories: + +- The `.dagger` directory will store metadata about _environments_, _inputs_, and _outputs_ which we will cover later. - The `cue.mod` directory stores libraries such as [dagger/universe](https://github.com/dagger/universe) which can be _imported_ into your Dagger _plan_. Dagger will load all `.cue` files recursively in the current Dagger project. More directories can be added to help organize code. @@ -53,54 +65,155 @@ Dagger will load all `.cue` files recursively in the current Dagger project. Mor ### Write a Dagger Plan -A Dagger _plan_ is written in CUE and expresses the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. Unlike traditional glue code written in an scripting language (e.g.: Bash, PowerShell), a Dagger plan is _declarative_ rather than _imperative_. This frees us from thinking about order of operations, since Dagger will infer dependendencies and calculate correct order on its own. +A Dagger _plan_ is written in CUE and defines the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. Unlike traditional glue code written in a scripting language such as Bash or PowerShell, a Dagger plan is _declarative_ rather than _imperative_. This frees us from thinking about the order of operations, since Dagger will infer dependendencies and calculate correct order on its own. -First create a directory to hold our plan, separate from our application code: +Let's first create a directory to hold our plan separately from our application code: -```shell -mkdir ./plan +```bash +mkdir -p ./plans/local ``` -Next, create a file in `plan/` called `todoapp.cue` with the following content +We will now create the following files: + +- `plans/todoapp.cue` which will define resources common to all environments +- `plans/local/local.cue` which will define resources specific to the local environment + +Create the file `plans/todoapp.cue` with the following content: ```cue package todoapp import ( - "alpha.dagger.io/dagger" - "alpha.dagger.io/dagger/stream" - "alpha.dagger.io/js/yarn" + "alpha.dagger.io/dagger" + "alpha.dagger.io/os" + "alpha.dagger.io/docker" + "alpha.dagger.io/js/yarn" ) -// Source code of the sample application -source: dagger.#Artifact & dagger.#Input - // Build the source code using Yarn app: yarn.#Package & { - "source": source + "source": dagger.#Artifact & dagger.#Input +} + +// package the staic HTML from yarn into a Docker image +image: os.#Container & { + image: docker.#Pull & { + from: "nginx" + } + + copy: "/usr/share/nginx/html": from: app.build +} + +// push the image to a registry +push: docker.#Push & { + // leave target as a string here so that + // different environments can push to different registries + target: string + source: image } ``` +This file will define the resources and relationships between them that are common across all environments. For example here we are deploying to our local Docker engine in our `local` environment, but for staging or production as examples, we would deploy to some other container orchestration system such as Kubernetes somewhere out there among the various cloud providers. + +Create the file `plans/local/local.cue` with the following content: + +```cue +package todoapp + +import ( + "alpha.dagger.io/dagger" + "alpha.dagger.io/docker" +) + +// run our todoapp in our local Docker engine +todoapp: docker.#Run & { + ref: push.ref + name: "todoapp" + ports: ["8080:80"] + socket: dagger.#Stream & dagger.#Input +} + +// push to our local registry +push: target: "localhost:5000/todoapp" +``` + +Notice that both files have the same `package todoapp` declared on the first line. This is crucial to inform CUE that they are to be loaded and evaluated together in the same context. + +Our `local.cue` file now holds resources specific to our `local` environment. Also notice that we are defining a concrete value for the `target` key here. The entire `push` object is defined in both files and CUE will merge the values found among our 2 files. + ### Create an Environment +Before we can deploy the plan, we need to define an environment which is the specific plan to execute, as well as the context from which inputs are pulled and to which state is stored. + +In this example we will deploy the app to our local docker engine so let’s create a `local` environment: + ```shell -dagger new local -p ./plan +dagger new local -p ./plans/local dagger list ``` +The `list` command shows the current environments defined: + +```bash +local ...todoapp/.dagger/env/local +``` + ### Define Input Values per Environment -```shell -dagger input list -``` +Our Dagger plan includes a number of references to `dagger.#Input` which inform the Dagger engine that the concrete value should be pulled from inputs. While some things such as the registry target we saw above can be expressed purely in CUE, others such as directories, secrets, and sockets are required to be explicitly defined as _inputs_ to ensure security. If Dagger allowed such things to be stated in CUE, the entire package system could become a source of attacks. -```text -Input Value Set by user Description -app.source dagger.#Artifact false Application source code -``` +List the inputs Dagger is aware of according to our plan: ```shell -dagger -e local input dir app.source ./app +dagger -e local input list ``` +You should see the following output: + +```bash +Input Value Set by user Description +app.source dagger.#Artifact false Application source code +todoapp.socket struct false Mount local docker socket +``` + +Notice that `Set by user` is false for both, because we have not yet provided Dagger with those values. + +Let’s provide them now: + +```shell +dagger -e local input socket todoapp.socket /var/run/docker.sock +dagger -e local input dir app.source ./ + +``` + +Now let's replay the `dagger input list` command: + +```bash +Input Value Set by user Description +app.source dagger.#Artifact true Application source code +todoapp.socket struct true Mount local docker socket +``` + +Notice that Dagger now reports that both inputs have been set. + +### Deploy the Appplication + +With our plan in place, our environment set, and our inputs defined we can deploy the application: + +```bash +dagger up +``` + +Once complete you should get logs, and a final output like this: + +```bash +Output Value Description +app.build struct Build output directory +push.ref "localhost:5000/todoapp:latest@sha256:" Image ref +push.digest "sha256:" Image digest +todoapp.ref "localhost:5000/todoapp:latest@sha256:" Image reference (e.g: nginx:alpine) +todoapp.run.env.IMAGE_REF "localhost:5000/todoapp:latest@sha256:" - +``` + +Congratulations! You’ve deployed your first Dagger plan! You can now open [http://localhost:8080](http://localhost:8080) in your browser! From b78169ed3f311a0a92ce099f91f8fa1a9d3c74b1 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Fri, 1 Oct 2021 15:25:45 -0600 Subject: [PATCH 04/10] typos, comments, and clarifications Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index ef5c31ce..f2cedf10 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -4,7 +4,7 @@ slug: /1003/get-started/ # Get Started with Dagger -In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple project deploys a [React](https://reactjs.org/) application to your local machine via docker. In later tutorials, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, how to share access to your infrastructure in the same way that we share access to ours now. +In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple project deploys a [React](https://reactjs.org/) application to your local machine via Docker. In later tutorials, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, how to share access to your infrastructure in the same way that we share access to ours now. This tutorial does involve writing CUE, so if you haven’t already, be sure to read [What is CUE?](../introduction/1005-what_is_cue.md) @@ -47,7 +47,7 @@ drwxr-xr-x ... 192 Sep 29 11:17 src -rw-r--r-- ... 465514 Sep 29 11:17 yarn.lock ``` -Now we need to initialize this directory as a Dagger _project_ (and relist directories): +Now we need to initialize this directory as a Dagger _project_: ```bash dagger init @@ -95,12 +95,15 @@ app: yarn.#Package & { "source": dagger.#Artifact & dagger.#Input } -// package the staic HTML from yarn into a Docker image +// package the static HTML from yarn into a Docker image image: os.#Container & { image: docker.#Pull & { from: "nginx" } + // app.build references our app key above + // which infers a dependency that Dagger + // uses to generate the DAG copy: "/usr/share/nginx/html": from: app.build } @@ -109,12 +112,15 @@ push: docker.#Push & { // leave target as a string here so that // different environments can push to different registries target: string + + // the source of our push resource + // is the image resource we declared above source: image } ``` -This file will define the resources and relationships between them that are common across all environments. For example here we are deploying to our local Docker engine in our `local` environment, but for staging or production as examples, we would deploy to some other container orchestration system such as Kubernetes somewhere out there among the various cloud providers. +This file will define the resources and relationships between them that are common across _all environments_. For example, here we are deploying to our local Docker engine in our `local` environment, but for staging or production as examples, we would deploy the same image to some other container orchestration system such as Kubernetes hosted somewhere out there among the various cloud providers. Create the file `plans/local/local.cue` with the following content: @@ -140,13 +146,13 @@ push: target: "localhost:5000/todoapp" Notice that both files have the same `package todoapp` declared on the first line. This is crucial to inform CUE that they are to be loaded and evaluated together in the same context. -Our `local.cue` file now holds resources specific to our `local` environment. Also notice that we are defining a concrete value for the `target` key here. The entire `push` object is defined in both files and CUE will merge the values found among our 2 files. +Our `local.cue` file now holds resources specific to our `local` environment. Also notice that we are defining a concrete value for the `target` key here. The entire `push` object is defined in both files and CUE will merge the values into a single struct with key:value pairs that are _complete_ with concrete values. ### Create an Environment Before we can deploy the plan, we need to define an environment which is the specific plan to execute, as well as the context from which inputs are pulled and to which state is stored. -In this example we will deploy the app to our local docker engine so let’s create a `local` environment: +For our each environment we need to tell Dagger what CUE files to load, so let’s create a `local` environment: ```shell dagger new local -p ./plans/local @@ -161,7 +167,7 @@ local ...todoapp/.dagger/env/local ### Define Input Values per Environment -Our Dagger plan includes a number of references to `dagger.#Input` which inform the Dagger engine that the concrete value should be pulled from inputs. While some things such as the registry target we saw above can be expressed purely in CUE, others such as directories, secrets, and sockets are required to be explicitly defined as _inputs_ to ensure security. If Dagger allowed such things to be stated in CUE, the entire package system could become a source of attacks. +Our Dagger plan includes a number of references to `dagger.#Input` which inform the Dagger engine that the concrete value should be pulled from inputs. While some things such as the registry target we saw above can be expressed purely in CUE, others such as directories, secrets, and sockets are required to be explicitly defined as _inputs_ to protect against malicious code being injected by third-party packages. If Dagger allowed such things to be stated in CUE, the entire package system could become a source of attacks. List the inputs Dagger is aware of according to our plan: @@ -187,6 +193,8 @@ dagger -e local input dir app.source ./ ``` +This defines the `todoapp.socket` as a `socket` input type, and the `app.source` input as a `dir` input type. + Now let's replay the `dagger input list` command: ```bash @@ -199,7 +207,7 @@ Notice that Dagger now reports that both inputs have been set. ### Deploy the Appplication -With our plan in place, our environment set, and our inputs defined we can deploy the application: +With our plan in place, our environment set, and our inputs defined, we can deploy the application as simply as: ```bash dagger up From 46c02c7563361b796772460355738af55e898226 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Mon, 4 Oct 2021 09:59:14 -0600 Subject: [PATCH 05/10] proofreading corrections Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 52 ++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 25 deletions(-) diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index f2cedf10..c3fa1a3d 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -4,7 +4,7 @@ slug: /1003/get-started/ # Get Started with Dagger -In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple project deploys a [React](https://reactjs.org/) application to your local machine via Docker. In later tutorials, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, how to share access to your infrastructure in the same way that we share access to ours now. +In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple project deploys a [React](https://reactjs.org/) application to your local machine via Docker. In later tutorials, you will learn how to configure Dagger to deploy to remote infrastructure such as EKS and GKE. This tutorial does involve writing CUE, so if you haven’t already, be sure to read [What is CUE?](../introduction/1005-what_is_cue.md) @@ -16,7 +16,7 @@ In this tutorial we will learn: - environments - inputs and outputs - How to write CUE for Dagger -- How to deploy an application using `dagger up` +- How to deploy an application with Dagger ## Deploy an Application Locally @@ -57,7 +57,7 @@ ls -la You will now see 2 new directories: - The `.dagger` directory will store metadata about _environments_, _inputs_, and _outputs_ which we will cover later. -- The `cue.mod` directory stores libraries such as [dagger/universe](https://github.com/dagger/universe) which can be _imported_ into your Dagger _plan_. +- The `cue.mod` directory stores libraries such as [dagger/universe](https://github.com/dagger/universe) which can be _imported_ into your Dagger plan. Dagger will load all `.cue` files recursively in the current Dagger project. More directories can be added to help organize code. @@ -67,7 +67,7 @@ Dagger will load all `.cue` files recursively in the current Dagger project. Mor A Dagger _plan_ is written in CUE and defines the _resources_, _dependencies_, and _logic_ to deploy an application to an environment. Unlike traditional glue code written in a scripting language such as Bash or PowerShell, a Dagger plan is _declarative_ rather than _imperative_. This frees us from thinking about the order of operations, since Dagger will infer dependendencies and calculate correct order on its own. -Let's first create a directory to hold our plan separately from our application code: +Let’s first create a directory to hold our Dagger plan separately from our application code: ```bash mkdir -p ./plans/local @@ -109,8 +109,8 @@ image: os.#Container & { // push the image to a registry push: docker.#Push & { - // leave target as a string here so that - // different environments can push to different registries + // leave target blank here so that different + // environments can push to different registries target: string // the source of our push resource @@ -133,7 +133,7 @@ import ( ) // run our todoapp in our local Docker engine -todoapp: docker.#Run & { +run: docker.#Run & { ref: push.ref name: "todoapp" ports: ["8080:80"] @@ -141,6 +141,8 @@ todoapp: docker.#Run & { } // push to our local registry +// this concrete value satisfies the string constraint +// we defined in the previous file push: target: "localhost:5000/todoapp" ``` @@ -167,7 +169,7 @@ local ...todoapp/.dagger/env/local ### Define Input Values per Environment -Our Dagger plan includes a number of references to `dagger.#Input` which inform the Dagger engine that the concrete value should be pulled from inputs. While some things such as the registry target we saw above can be expressed purely in CUE, others such as directories, secrets, and sockets are required to be explicitly defined as _inputs_ to protect against malicious code being injected by third-party packages. If Dagger allowed such things to be stated in CUE, the entire package system could become a source of attacks. +Our Dagger plan includes a number of references to `dagger.#Input` which inform the Dagger engine that the concrete value should be pulled from inputs at runtime. While some values such as the registry target we saw above can be expressed purely in CUE, others such as directories, secrets, and sockets are required to be explicitly defined as _inputs_ to protect against malicious code being injected by third-party packages. If Dagger allowed such things to be stated in CUE, the entire package system could become a source of attacks. List the inputs Dagger is aware of according to our plan: @@ -178,29 +180,29 @@ dagger -e local input list You should see the following output: ```bash -Input Value Set by user Description -app.source dagger.#Artifact false Application source code -todoapp.socket struct false Mount local docker socket +Input Value Set by user Description +app.source dagger.#Artifact false Application source code +run.socket struct false Mount local docker socket ``` -Notice that `Set by user` is false for both, because we have not yet provided Dagger with those values. +Notice that `Set by user` is _false_ for both, because we have not yet provided Dagger with those values. Let’s provide them now: ```shell -dagger -e local input socket todoapp.socket /var/run/docker.sock +dagger -e local input socket run.socket /var/run/docker.sock dagger -e local input dir app.source ./ ``` -This defines the `todoapp.socket` as a `socket` input type, and the `app.source` input as a `dir` input type. +This defines the `run.socket` as a `socket` input type, and the `app.source` input as a `dir` input type. -Now let's replay the `dagger input list` command: +Now let’s replay the `dagger input list` command: ```bash -Input Value Set by user Description -app.source dagger.#Artifact true Application source code -todoapp.socket struct true Mount local docker socket +Input Value Set by user Description +app.source dagger.#Artifact true Application source code +run.socket struct true Mount local docker socket ``` Notice that Dagger now reports that both inputs have been set. @@ -216,12 +218,12 @@ dagger up Once complete you should get logs, and a final output like this: ```bash -Output Value Description -app.build struct Build output directory -push.ref "localhost:5000/todoapp:latest@sha256:" Image ref -push.digest "sha256:" Image digest -todoapp.ref "localhost:5000/todoapp:latest@sha256:" Image reference (e.g: nginx:alpine) -todoapp.run.env.IMAGE_REF "localhost:5000/todoapp:latest@sha256:" - +Output Value Description +app.build struct Build output directory +push.ref "localhost:5000/todoapp:latest@sha256:" Image ref +push.digest "sha256:" Image digest +run.ref "localhost:5000/todoapp:latest@sha256:" Image reference (e.g: nginx:alpine) +run.run.env.IMAGE_REF "localhost:5000/todoapp:latest@sha256:" - ``` -Congratulations! You’ve deployed your first Dagger plan! You can now open [http://localhost:8080](http://localhost:8080) in your browser! +Congratulations! You’ve deployed your first Dagger plan! You can now [view the todo app](http://localhost:8080) in your browser! From 35a86441bf2a751e6eb417b6f8ee47a41cd58269 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Tue, 5 Oct 2021 11:57:21 -0600 Subject: [PATCH 06/10] added test Signed-off-by: Richard Jones --- docs/learn/1003-get-started.md | 59 +------- docs/learn/1003-get-started.md.bak | 126 ++++++++++++++++++ docs/learn/tests/doc.bats | 26 ++-- .../getting-started/plans/local/local.cue | 19 +++ .../tests/getting-started/plans/todoapp.cue | 36 +++++ 5 files changed, 195 insertions(+), 71 deletions(-) create mode 100644 docs/learn/1003-get-started.md.bak create mode 100644 docs/learn/tests/getting-started/plans/local/local.cue create mode 100644 docs/learn/tests/getting-started/plans/todoapp.cue diff --git a/docs/learn/1003-get-started.md b/docs/learn/1003-get-started.md index c3fa1a3d..0f6e493d 100644 --- a/docs/learn/1003-get-started.md +++ b/docs/learn/1003-get-started.md @@ -80,70 +80,15 @@ We will now create the following files: Create the file `plans/todoapp.cue` with the following content: -```cue -package todoapp - -import ( - "alpha.dagger.io/dagger" - "alpha.dagger.io/os" - "alpha.dagger.io/docker" - "alpha.dagger.io/js/yarn" -) - -// Build the source code using Yarn -app: yarn.#Package & { - "source": dagger.#Artifact & dagger.#Input -} - -// package the static HTML from yarn into a Docker image -image: os.#Container & { - image: docker.#Pull & { - from: "nginx" - } - - // app.build references our app key above - // which infers a dependency that Dagger - // uses to generate the DAG - copy: "/usr/share/nginx/html": from: app.build -} - -// push the image to a registry -push: docker.#Push & { - // leave target blank here so that different - // environments can push to different registries - target: string - - // the source of our push resource - // is the image resource we declared above - source: image -} - +```cue file=./tests/getting-started/plans/todoapp.cue ``` This file will define the resources and relationships between them that are common across _all environments_. For example, here we are deploying to our local Docker engine in our `local` environment, but for staging or production as examples, we would deploy the same image to some other container orchestration system such as Kubernetes hosted somewhere out there among the various cloud providers. Create the file `plans/local/local.cue` with the following content: -```cue -package todoapp +```cue file=./tests/getting-started/plans/local/local.cue -import ( - "alpha.dagger.io/dagger" - "alpha.dagger.io/docker" -) - -// run our todoapp in our local Docker engine -run: docker.#Run & { - ref: push.ref - name: "todoapp" - ports: ["8080:80"] - socket: dagger.#Stream & dagger.#Input -} - -// push to our local registry -// this concrete value satisfies the string constraint -// we defined in the previous file -push: target: "localhost:5000/todoapp" ``` Notice that both files have the same `package todoapp` declared on the first line. This is crucial to inform CUE that they are to be loaded and evaluated together in the same context. diff --git a/docs/learn/1003-get-started.md.bak b/docs/learn/1003-get-started.md.bak new file mode 100644 index 00000000..f5eaf5af --- /dev/null +++ b/docs/learn/1003-get-started.md.bak @@ -0,0 +1,126 @@ +--- +slug: /1003/get-started/ +--- + +# Get Started with Dagger + +In this tutorial, you will learn the basics of Dagger by building a Dagger project from scratch. This simple +project deploys a [React](https://reactjs.org/) application to your local machine via docker. In later tutorials, +you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, +how to share access to your infrastructure in the same way that we share access to ours now. + +This tutorial does involve writing CUE, so if you haven’t already, be sure to read [What is CUE?](../introduction/1005-what_is_cue.md) + +In this tutorial we will learn how to: +- Structure a Dagger project +- Write CUE for Dagger +- Deploy an application using Dagger + +## Initial setup + +### Install Dagger + +First, make sure [you have installed Dagger on your local machine](../1001-install.md). + +### Setup example app + +You will need a local copy of the [Dagger examples repository](https://github.com/dagger/examples). +NOTE: you may use the same local copy across all tutorials. + +```shell +git clone https://github.com/dagger/examples +``` + +Make sure that all commands are run from the `todoapp` directory: + +```shell +cd examples/todoapp +``` + +### Import the tutorial key + +Dagger natively supports encrypted secrets: when a user inputs a value marked as secret +(for example, a password, API token, or ssh key) it is automatically encrypted with that user's key, +and no other user can access that value unless they are explicitly given access. + +In the interest of security, Dagger has no way _not_ to encrypt a secret value. +But this causes a dilemma for this tutorial: how do we give unrestricted, public access to our +(carefully sandboxed) infrastructure so that anyone can deploy to it? + +To solve this dilemma, we included the private key used to encrypt the tutorial's secret inputs. +Import the key to your Dagger installation, and you're good to go: + +```shell +./import-tutorial-key.sh +``` + +## First deployment + +Now that your environment is set up, you are ready to deploy: + +```shell +dagger up +``` + +That's it! You have just made your first deployment with Dagger. + +The URL of your newly deployed app should be visible towards the end of the command output. +If you visit that URL, you should see your application live! + +## Code, deploy, repeat + +This environment is pre-configured to deploy from the `./todoapp` directory, +so you can make any change you want to that directory, then deploy it with `dagger up`. +You can even replace our example React code with any React application! + +NOTE: you don't have to commit your changes to the git repository before deploying them. + +## Under the hood + +This example showed you how to deploy and develop an application that is already configured with Dagger. Now, let's learn a few concepts to help you understand how this was put together. + +### The Environment + +An Environment holds the entire deployment configuration. + +You can list existing environment from the `./todoapp` directory: + +```shell +dagger list +``` + +You should see an environment named `s3`. You can have many environments within your app. For instance, one for `staging`, one for `dev`, etc... + +Each environment can have a different kind of deployment code. For example, a `dev` environment can deploy locally; a `staging` environment can deploy to a remote infrastructure, and so on. + +### The plan + +The plan is the deployment code that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan: + +```shell +ls -l ./s3 +``` + +Any code change to the plan will be applied during the next `dagger up`. + +### The inputs + +The plan can define one or several `inputs`. Inputs may be configuration values, artifacts, or encrypted secrets provided by the user. Here is how to list the current inputs: + +```shell +dagger input list +``` + +The inputs are persisted inside the `.dagger` directory and pushed to your git repository. That's why this example application worked out of the box. + +### The outputs + +The plan defines one or several `outputs`. They can show helpful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all outputs: + +```shell +dagger output list +``` + +## What's next? + +At this point, you have deployed your first application using Dagger and learned some dagger commands. You are now ready to [learn more about how to program Dagger](./1004-first-env.md). diff --git a/docs/learn/tests/doc.bats b/docs/learn/tests/doc.bats index 1cebc557..da94141c 100644 --- a/docs/learn/tests/doc.bats +++ b/docs/learn/tests/doc.bats @@ -11,21 +11,19 @@ setup() { @test "doc-1003-get-started" { setup_example_sandbox - # Set examples private key - "$DAGGER_SANDBOX"/import-tutorial-key.sh + # Follow tutorial + mkdir -p "$DAGGER_SANDBOX"/getting-started/plans + cp -R "$DAGGER_PROJECT"/getting-started/ "$DAGGER_SANDBOX"/getting-started/ + + dagger --project "$DAGGER_SANDBOX" new 'local' -p "$DAGGER_SANDBOX"/getting-started/plans/local + dagger --project "$DAGGER_SANDBOX" -e 'local' input socket run.socket /var/run/docker.sock + dagger --project "$DAGGER_SANDBOX" -e 'local' input dir app.source "$DAGGER_SANDBOX" + dagger --project "$DAGGER_SANDBOX" -e 'local' up - # Collect url - dagger --project "$DAGGER_SANDBOX" up - url=$(dagger --project "$DAGGER_SANDBOX" query -f text url) - - # More commands - dagger --project "$DAGGER_SANDBOX" list - ls -l "$DAGGER_SANDBOX"/s3 - dagger --project "$DAGGER_SANDBOX" input list - - # Check output - run curl "$url" - assert_output --partial "My Todo app" + until docker inspect --format "{{json .State.Status }}" todoapp | grep -m 1 "running"; do sleep 1 ; done + run curl -f -LI http://localhost:8080 + assert_output --partial '200 OK' + docker stop todoapp && docker rm todoapp } @test "doc-1004-first-env" { diff --git a/docs/learn/tests/getting-started/plans/local/local.cue b/docs/learn/tests/getting-started/plans/local/local.cue new file mode 100644 index 00000000..081df547 --- /dev/null +++ b/docs/learn/tests/getting-started/plans/local/local.cue @@ -0,0 +1,19 @@ +package todoapp + +import ( + "alpha.dagger.io/dagger" + "alpha.dagger.io/docker" +) + +// run our todoapp in our local Docker engine +run: docker.#Run & { + ref: push.ref + name: "todoapp" + ports: ["8080:80"] + socket: dagger.#Stream & dagger.#Input +} + +// push to our local registry +// this concrete value satisfies the string constraint +// we defined in the previous file +push: target: "localhost:5000/todoapp" \ No newline at end of file diff --git a/docs/learn/tests/getting-started/plans/todoapp.cue b/docs/learn/tests/getting-started/plans/todoapp.cue new file mode 100644 index 00000000..867aeb5e --- /dev/null +++ b/docs/learn/tests/getting-started/plans/todoapp.cue @@ -0,0 +1,36 @@ +package todoapp + +import ( + "alpha.dagger.io/dagger" + "alpha.dagger.io/os" + "alpha.dagger.io/docker" + "alpha.dagger.io/js/yarn" +) + +// Build the source code using Yarn +app: yarn.#Package & { + "source": dagger.#Artifact & dagger.#Input +} + +// package the static HTML from yarn into a Docker image +image: os.#Container & { + image: docker.#Pull & { + from: "nginx" + } + + // app.build references our app key above + // which infers a dependency that Dagger + // uses to generate the DAG + copy: "/usr/share/nginx/html": from: app.build +} + +// push the image to a registry +push: docker.#Push & { + // leave target blank here so that different + // environments can push to different registries + target: string + + // the source of our push resource + // is the image resource we declared above + source: image +} \ No newline at end of file From 4c7d90f9bc752b9c30d0e38ba5029f2a43a4b068 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Tue, 5 Oct 2021 12:04:10 -0600 Subject: [PATCH 07/10] cue fmt Signed-off-by: Richard Jones --- .../getting-started/plans/local/local.cue | 14 +++---- .../tests/getting-started/plans/todoapp.cue | 38 +++++++++---------- 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/docs/learn/tests/getting-started/plans/local/local.cue b/docs/learn/tests/getting-started/plans/local/local.cue index 081df547..590acffd 100644 --- a/docs/learn/tests/getting-started/plans/local/local.cue +++ b/docs/learn/tests/getting-started/plans/local/local.cue @@ -1,19 +1,19 @@ package todoapp import ( - "alpha.dagger.io/dagger" - "alpha.dagger.io/docker" + "alpha.dagger.io/dagger" + "alpha.dagger.io/docker" ) // run our todoapp in our local Docker engine run: docker.#Run & { - ref: push.ref - name: "todoapp" - ports: ["8080:80"] - socket: dagger.#Stream & dagger.#Input + ref: push.ref + name: "todoapp" + ports: ["8080:80"] + socket: dagger.#Stream & dagger.#Input } // push to our local registry // this concrete value satisfies the string constraint // we defined in the previous file -push: target: "localhost:5000/todoapp" \ No newline at end of file +push: target: "localhost:5000/todoapp" diff --git a/docs/learn/tests/getting-started/plans/todoapp.cue b/docs/learn/tests/getting-started/plans/todoapp.cue index 867aeb5e..b62d93d2 100644 --- a/docs/learn/tests/getting-started/plans/todoapp.cue +++ b/docs/learn/tests/getting-started/plans/todoapp.cue @@ -1,36 +1,36 @@ package todoapp import ( - "alpha.dagger.io/dagger" - "alpha.dagger.io/os" - "alpha.dagger.io/docker" - "alpha.dagger.io/js/yarn" + "alpha.dagger.io/dagger" + "alpha.dagger.io/os" + "alpha.dagger.io/docker" + "alpha.dagger.io/js/yarn" ) // Build the source code using Yarn app: yarn.#Package & { - "source": dagger.#Artifact & dagger.#Input + "source": dagger.#Artifact & dagger.#Input } // package the static HTML from yarn into a Docker image image: os.#Container & { - image: docker.#Pull & { - from: "nginx" - } + image: docker.#Pull & { + from: "nginx" + } - // app.build references our app key above - // which infers a dependency that Dagger - // uses to generate the DAG - copy: "/usr/share/nginx/html": from: app.build + // app.build references our app key above + // which infers a dependency that Dagger + // uses to generate the DAG + copy: "/usr/share/nginx/html": from: app.build } // push the image to a registry push: docker.#Push & { - // leave target blank here so that different - // environments can push to different registries - target: string + // leave target blank here so that different + // environments can push to different registries + target: string - // the source of our push resource - // is the image resource we declared above - source: image -} \ No newline at end of file + // the source of our push resource + // is the image resource we declared above + source: image +} From 79489ffe3694b9bc099bca10830e33519a545c04 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Tue, 5 Oct 2021 12:08:44 -0600 Subject: [PATCH 08/10] cue fmt the right way Signed-off-by: Richard Jones --- docs/learn/tests/getting-started/plans/todoapp.cue | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/learn/tests/getting-started/plans/todoapp.cue b/docs/learn/tests/getting-started/plans/todoapp.cue index b62d93d2..8fa8b0cd 100644 --- a/docs/learn/tests/getting-started/plans/todoapp.cue +++ b/docs/learn/tests/getting-started/plans/todoapp.cue @@ -9,7 +9,7 @@ import ( // Build the source code using Yarn app: yarn.#Package & { - "source": dagger.#Artifact & dagger.#Input + source: dagger.#Artifact & dagger.#Input } // package the static HTML from yarn into a Docker image From f48d145f036c0c72af5b7baa26d43b6adea3469d Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Tue, 5 Oct 2021 12:40:39 -0600 Subject: [PATCH 09/10] added registry to test, and fixed cp commands Signed-off-by: Richard Jones --- docs/learn/tests/doc.bats | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/learn/tests/doc.bats b/docs/learn/tests/doc.bats index da94141c..ecac9f2a 100644 --- a/docs/learn/tests/doc.bats +++ b/docs/learn/tests/doc.bats @@ -12,18 +12,23 @@ setup() { setup_example_sandbox # Follow tutorial - mkdir -p "$DAGGER_SANDBOX"/getting-started/plans - cp -R "$DAGGER_PROJECT"/getting-started/ "$DAGGER_SANDBOX"/getting-started/ + mkdir -p "$DAGGER_SANDBOX"/plans/local + cp "$DAGGER_PROJECT"/getting-started/plans/todoapp.cue "$DAGGER_SANDBOX"/plans/todoapp.cue + cp "$DAGGER_PROJECT"/getting-started/plans/local/local.cue "$DAGGER_SANDBOX"/plans/local/local.cue - dagger --project "$DAGGER_SANDBOX" new 'local' -p "$DAGGER_SANDBOX"/getting-started/plans/local + dagger --project "$DAGGER_SANDBOX" new 'local' -p "$DAGGER_SANDBOX"/plans/local dagger --project "$DAGGER_SANDBOX" -e 'local' input socket run.socket /var/run/docker.sock dagger --project "$DAGGER_SANDBOX" -e 'local' input dir app.source "$DAGGER_SANDBOX" + + docker run -d -p 5000:5000 --name registry registry:2 + dagger --project "$DAGGER_SANDBOX" -e 'local' up until docker inspect --format "{{json .State.Status }}" todoapp | grep -m 1 "running"; do sleep 1 ; done run curl -f -LI http://localhost:8080 assert_output --partial '200 OK' docker stop todoapp && docker rm todoapp + docker stop registry && docker rm registry } @test "doc-1004-first-env" { From 7c55d386259bb4cbc551816d3aad225f4e9f90f1 Mon Sep 17 00:00:00 2001 From: Richard Jones Date: Tue, 5 Oct 2021 12:53:34 -0600 Subject: [PATCH 10/10] removed registry Signed-off-by: Richard Jones --- docs/learn/tests/doc.bats | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/learn/tests/doc.bats b/docs/learn/tests/doc.bats index ecac9f2a..9b3af3b1 100644 --- a/docs/learn/tests/doc.bats +++ b/docs/learn/tests/doc.bats @@ -20,8 +20,6 @@ setup() { dagger --project "$DAGGER_SANDBOX" -e 'local' input socket run.socket /var/run/docker.sock dagger --project "$DAGGER_SANDBOX" -e 'local' input dir app.source "$DAGGER_SANDBOX" - docker run -d -p 5000:5000 --name registry registry:2 - dagger --project "$DAGGER_SANDBOX" -e 'local' up until docker inspect --format "{{json .State.Status }}" todoapp | grep -m 1 "running"; do sleep 1 ; done