These days software development is about far more than just writing software.
Your average developer needs to know everything from setting up database services through to deploying the systems they create to a set of production servers, no longer do developers get by with just writing code and throwing it over the wall for someone else to deal with.
This is actually a very good thing, because it means that the developers of software are involved in its entire life cycle. Faults and issues are often caught and resolved before anything goes into production, and a multi skilled team of developers, testers, DBA's and server admins working together in harmony means everyone learns from each other.
There is unfortunately however, also a dark side to this multi skill idea, and that's the concept that, there are a minority of businesses that now believe they don't need to employ a full I.T. team to run their projects, and can get by just employing a single developer who can do everything.
Even in a good company who understands the developers job, there are time constraints, targets to be met and all manner of other things which lead us to be not as effective as we might like to be at covering all the bases.
As good I.T. professionals we therefore look to using our skills sets to automate as much of our work as we can, and one of the areas where this is very prevalent, is in the arena of building and testing our software
Bring in the butler
If you never heard of "Jenkins CI" before now, then you've never experienced a "Digital Butler" before :)
Jenkins quite simply is an "Automation" system of sorts, primarily aimed at automating software builds and deployments, it can actually be used for a great many other tasks. I for example use it to upload manuscripts for blogs and books I write directly to things like my Google Drive and MS-One drives for backup purposes.
In this series of 4 posts I'm going to show you how to set up and run a simple Virtual Machine based system that can take your Dot Net core & .NET 5 projects directly from your source control service, build them, test that build and then deploy the result to a staging server.
Once you understand the basics of setting up a simple CI server to build your software, you can then start using the techniques you learn to take your builds to the next level.
What we'll be building
We'll be setting up an Ubuntu Linux server in a Virtual Machine environment, this environment will periodically check a project checked into a Git Source control server (We won't be using GitHub in this post, but the steps where we use normal Git can easily be swapped for a Github account), the project will be checked out, and built using the dotnet core command line tool set, we'll then run a quick check on the app we built to test that it was started correctly.
Along the way we'll learn how to make Jenkins run supervisor/root only actions on a Linux server, and how to register our service as a Linux service that starts automatically at boot time.
In order to do this, you will need access to the ability to run a virtual server. If you have a modern day Windows 10 install, you should be able to you use the built in Hyper-V console to create a local VM machine.
Failing that, Virtual Box will allow you to easily set up a system, or if you have an account with a provider such as Linode which gives you an SSD Based VM to provision as you see fit, that will do the trick just as well.
In preparation for creating the VM, you'll need to download a suitable Ubuntu Linux Server ISO from here: http://releases.ubuntu.com/18.04.4/
At the time of writing this, Ubuntu versions 19 & 20 are also now available, but as 18.4 is still in Long Term Support and that’s the version I'm used to using, that's the version this series of blog posts will be based around.
Depending on when you read this post, future versions may be different.
What constitutes a Dot Net Build
Let's quickly remind ourselves what's involved in a dot net command line build.
In most cases it's simply
-
Dotnet Restore
-
Dotnet Build
-
Dotnet Publish
-
Dotnet Run
The last one is only needed if you're running your app immediately after building it. In most cases you'll copy the app to a known location, then configure something else to launch it.
In the case of an Ubuntu system this will be the "SystemD" boot service, configured to start your Dotnet Kestrel service so that it starts when the Linux server is booted.
The above steps may seem quite simple, but there are a number of other factors to consider.
In Particular, you generally don't expose your Dotnet app directly to the public IP address of your server. Instead we generally forward proxy the traffic into the service which is only listening on "Localhost" for security purposes, the forward proxying is typically performed by dedicated web server software such as Nginx, Apache or in the case of Windows servers IIS.
As part of our build we'll also make a request to the "index" page of our .NET service, and if this page can be retrieved, we'll consider our service running and active.
After the Dotnet publish step, we'll also need to move our built application from the build location the location where it will be run from, in a Linux based server this will very often involve setting up permissions and users on the file system, whom have appropriate privileges to perform a given task in the system.
This permission system is very unlike Windows where you have a single "Would you like to run this as an Administrator" prompt when a task requires to be run with elevated privileges.
On a Linux system, there are multiple users, and each user has areas of the file system that belong to them and them alone. For a standard user, this is the user's home directory, for an application such as a web-server, this will be the folders where their web pages and log files are stored.
To deploy our Dotnet application, will likely mean not only copying it to its run location, but making sure that it has the correct privilege settings to allow it to run from the given location, and write files such as logs to the appropriate areas of the operating systems disk.
Basic VM setup
In preparation for part 2 where we will be setting up the VM, you'll need to set up Ubuntu ready to install the required software.
In MS Hyper-V (Which is what I'll be using) the minimum settings for the VM to be used look something like this:
Fig 2: Hyper-V Memory (ram) settings.
Fig 3: Hyper-V CPU Settings
Fig 4: Hyper-V Hard drive settings
Set your VM to boot from the ISO you downloaded from Ubuntu, and follow the instructions to install the OS when it boots.
It's also wise to allow your Ram amount to be elastic. 512Mb is a good starting amount, but since Jenkins runs on Java, this memory limit can and often does increase drastically.
On a Linux host, I've not had to allocate more than 2Gb of memory, on a windows host however, I've often had to allocate 4Gb, in the Ram settings for Hyper-V above however, I've set my maximum level to 10Gb.
The minimum size you need for your Hard Drive is 10Gb, but it's recommended to use a 30Gb hard disk, that way the OS has room to upgrade, and Jenkins will have space to set up temp builds. You may also want to install extras such as a Database server to go with your App, 30Gb is a good size to start any Linux install with.
While installing you will be asked a number of questions, one of which will be to add the "First System User", DO Not create a user called "root" or "Administrator", the OS installer will do this for you automatically. When you're asked to add a first user, this should be your regular user name and a password for you to use.
This user is important, as this will be the ONLY user by default that can make use of the "sudo" command to allow root/admin level operations to be performed on the system.
Lastly, as this will be a headless server, if you asked, you do not want to install a Desktop environment, if you are asked to install any extra base software, the only thing you'll need, will be "Open SSH Server" so that you can connect to your new VM using an app like Putty.
You shouldn't get asked any questions about firewall ports, but if you do, then the only ports you need to open are 8080 for Jenkins, 80 & 443 for the web server and 22 for your SSH server
In part 2, we'll see how to perform the main base software installation on your new VM, see you then.