AppDefinition🔗
Introduction🔗
This manual for the AppDefinition is made for developers with a solid experience with docker. Internally the SFH system uses kubernetes. As the added complexity of kubernetes might be overwhelming for most developers just wanting to create SFH apps, the process to create SFH apps is simplified and does not need knowledge of kubernetes and its specifics.
If you want to develop SFH apps and you have no docker experience up to this point, the SFH Development Team can give you a kickstart with your use case and also recommend good trainings for docker. Also if you want to dig deeper into kubernetes, trainings can be recommended
DeepDive into SFH Apps🔗
DISCLAIMER: If you just want to develop SFH Apps and you don't know what kubernetes is and you don't want to be confused by kubernetes technical jargon, you can safely skip this chapter.
Smart Factory Hosts are basically just worker nodes in a kubernetes cluster. And SFH Apps are helm charts that are installed in this kubernetes cluster with a node selector matching the target SFH. To reduce the complexity and reduce the set of pitholes for developers, SFH Apps must not and cannot be written as helm charts.
There is a SFH proprieritary JSON schema - called Appdefinition - that is inspired by kubernetes (k8s) and docker compose. The AppDefinition is then converted to a helm chart, when the AppVersion changes its state from development to testing. The generation of the Helm Charts is done by a component called AppGenerator which uses hygen as a templating engine.
An SFH App consist of one k8s deployment, which will create a single k8s Pod. All containers of the SFH App will be in this Pod.
Simple example🔗
Here is an example of an app with just one container and nothing else:
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "ubuntu",
"tag": "latest",
"name": "ubuntucontainer",
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi"
}
}
}
]
}
This example will just deploy a container, with an image ubuntu:latest. It won't be reachable from other apps or from the customers network. Also it will not be able to persist anything.
The container with the name ubuntucontainer has a guaranteed memory of 500MB but is also limited to 500MB memory. It requests just 1% time of one CPU, but on the other hand has no limitation and can use all remaining CPU on the SFH.
Resources🔗
It is required to defined a resources block for each container. The resources are defined exactly like in kubernetes workloads and the effects are exactly the same.
{
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi",
"cpu": "100m"
}
}
}
Memory Requests🔗
A container must request an amount of system memory, which it will use when running. This is the minimum of system memory, the container can use and which is guaranteed to be available for the container`s process. The process can use more than requested, but might expect to be killed by the OOM-Killer of the Linux HostOS, if the SFH is running out of memory.
It is recommended to set the memory request to the maximum memory footprint the container will have. On the other hand, the request should not be unreasonably high, as it will reduce the amount of other SFH Apps deployed to an SFH, as kubernetes will stop scheduling other apps to an SFH if the sum of memory requests is bigger than the memory available.
Memory Limits🔗
A container must define a limit for the system memory. If this limit is reached, the host will kill the container and the container will be restarted. Therefore the memory limit should be above the memory footprint the application reaches ever under normal circumstances. Therefore the memory limit will prevent damage by process with memory leaks or other defects.
The recommendation is to set the memory request and limit to the same value. For further explanation see: home.robusta.dev/blog/kubernetes-memory-limit
CPU Requests🔗
A container must request a portion of CPU. The value is given in "milliCPU" which represents thousandths of a CPU Core. This value will be used by the system to guarantee this container at least a time slice equivalent to the value. The CPU request should correspond to the normal CPU usage of the container. It should not be too small, as this could lead to throtling of the process below the demands of the process. Too big values should also be avoided, as it will reduce the amount of other SFH Apps deployed to an SFH, as kubernetes will stop scheduling other apps to an SFH if the sum of cpu requests is bigger than the cpus available.
CPU Limits🔗
CPU limits are optional. They can be defined and will then limit the maximum cpu usage of the container`s process. There are good reasons to not limit your container's cpu usage, as the Linux host OS is able to cope with multiple containers using the complete cpu time. For further explanation see: https://home.robusta.dev/blog/stop-using-cpu-limits
Networking🔗
In general SFH Apps can initiate connections outbound without any further configuration. If you want to expose services of your app, there are multiple possibilies to expose the container's ports:
- Communication between the app's containers: Apps can communicate between each other via localhost. Ports must be defined
- Communication between apps on a SFH: Apps can communicate to other apps on an SFH. aka App2App. Alternatively the communication between apps can be secure by the WorkloadAttestation feature
- Communication via SFH's host ports: Container ports can be exposed to SFH host ports.
- Communicaiton via SFH's reverse proxy: Container ports can be exposed via the SFH's reverse proxy
Headless Services: Stable app-to-app communication during internet downtimes🔗
To ensure stable app-to-app communication even if the connection to the Control Plane is lost, we also offer as alternative the communication through a headless service. For more information on headless services, please refer to the official-documentation.
To use this option, follow the instructions provided at the relevant paragraph.
Declaring a port🔗
The follwoing example just exposes port 80 to be available on localhost:80 for the "client" container. Therefore the container ports must be unique.
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80
}
]
},
{
"image": "client",
"tag": "latest",
"name": "client",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
}
}
]
}
App to App Communication🔗
The following example exposes the whoami's port 80 to other apps on the same SFH.
Headless Service: The ContainerPort is used for the communication instead of the app2appPort. BUT the app2appPort must still be set. It is best to simply specify the same port.
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"service": {
"app2appPort": 80
}
}
]
}
]
}
The port is available for other apps by adressing the name generated for this app - not the display name. An example for the adress is dgrand-jade-butterfly-18. The name is equal to the name of the harbor project and can be seen in the app's settings.
Headless Service: The address for the headless service is extended by “-headless”. In this case it would be dgrand-jade-butterfly-18-headless.
The App2AppPort doesn't have to be equal to the containerPort. An App2AppPort must be unique inside the app.
To restrict the access to selected apps the WorkloadAttestation proxies WorkloadAttestation can be used.
Host ports🔗
The following example exposes the whoami's port 80 to the sfh Host port 8080. Use host ports with caution and sparingly. Currently there are no technical precausions, that two apps claim to use the same host port. The first one deployed will win, the second app won't even run.
If your app want's to expose an http service, better use the reverse proxy.
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"hostPort": 8080
}
]
}
]
}
Reverse proxy🔗
The SFH comes with a reverse proxy. Apps can expose ports via this reverse proxy. Either they are exposed by a path prefix, a subdomain or both.
The reverse proxy also handles https termination with a valid certificate for a domain like <id>.sfh.edge-device.net. This DNS name will resolve to the LAN IP adress of the SFH.
Another feature of the reverse proxy is to handle client certificate authentication.
Basic settings🔗
The following example will expose the whoami's port 80 via the reverse proxy under the path <id>.sfh.edge-device.net/whoami and will be reachable via http:// and https://
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"service": {
"app2appPort": 80,
"reverseProxy": {
"httpProxy": true,
"httpsProxy": true,
"routerPath": "whoami",
"stripPrefix": false
}
}
}
]
}
]
}
The app2appPort property must be set, if the reverseProxy is enabled.
The stripPrefix property controls, wether the whoami container will be aware or not of the router prefix. Example: a http request at the reverse Proxy GET Z0001Z0001.sfh.edge-device.net/whoami/a/deep/path
stripPrefix:false: The whoami container get's the requestGET Z0001Z0001.sfh.edge-device.net/whoami/a/deep/pathforwardedstripPrefix:true: The whoami container get's the requestGET Z0001Z0001.sfh.edge-device.net/a/deep/pathforwarded
Hints regarding DNS name, HTTP and routing🔗
For technical reasons, a service exposed by https, can only be called by using the DNS name (<id>.sfh.edge-device.net). A service exposed by http, can also be called by using the ip adress of the sfh.
Although it is obvious, subDomain based routing can only be used by using the DNS name of the SFH.
The alternative to path based routing is subdomain routing. To expose the whoami service via whoami.<id>.sfh.edge-device.net, use the following example.
If http is set to false, calls via http will automatically redirected to https.
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"service": {
"app2appPort": 80,
"reverseProxy": {
"httpProxy": true,
"httpsProxy": true,
"subDomain": "whoami"
}
}
}
]
}
]
}
Authentication🔗
Currently the reverse proxy can be configured to verify client certificates. This feature can exlusively be used with subdomain based routing
Example:
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"service": {
"app2appPort": 80,
"reverseProxy": {
"httpProxy": true,
"httpsProxy": true,
"subDomain": "whoami",
"authentication": {
"verifyClientCertificate": {
"caCertificates": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUV0RENDQXB3Q0NRQ0lGVU1WR3VWSC9UQU5CZ2txaGtpRzl3MEJBUXNGQURBY01Sb3dHQVlEVlFRRERCRk4KZVNCRFpYSjBJRUYxZEdodmNtbDBlVEFlRncweU1qQTNNRFV4TXpFeU1EQmFGdzB5TXpBMk1qWXhNekV5TURCYQpNQnd4R2pBWUJnTlZCQU1NRVUxNUlFTmxjblFnUVhWMGFHOXlhWFI1TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGCkFBT0NBZzhBTUlJQ0NnS0NBZ0VBMEFkWkFoT1NiN0lrQkZXcmhrVGt2bjdjVGtnQmVpVlpON0JmM3FwMlVOT0cKVlBxdUdLclUyRnY3dUR2MzJvLzFCK3o4dks2SzZTMkFjV0wvVFl3SkpXWU9sRHpOWm9uSTF2RVJKWmVFL1VUawowMTdhV1BIbUdGZk05VVpFV3ZldHlRV1pwUmVYRWhRUWVxcUZlYUFidWFUWk9uKzF6TFJXOGFUQVlNQjRHNTNXCmMrVHJPTUhhd3NRVWNZL3NGN1JUbUZYNit6WmtJRCtjZ0J4RUdGR012b3psU29MdnI5THVqOUJxaHpCMnRXdlAKUEUrZHN2alFTbGJvR0FrRXl0alRER1Fick1uTTVIWEc1MHlENTJoUnFrR0Q1MVAzcU9abmRUcFdRL3p1TXZGOAoxcHUxM0ZvbERTd2U3ak9Nc1ZCeWhYRmRWMUpZbjFBTkhSSFgwdUZsOVVnNm1WY2dmU29pY1A1YnJRQytCV3ZlCmo5bkphYnh0N3BSQzJGY1lPQ3BlcUVLVFlsZDFRT3JpdzYwbXFVL1ZBdEdBaXFLNmRjaFJFbUdMNk1QeTJYMCsKYXhrSHd5L3lGZ3lZRC94dCtYQ0MwRUZmY1grNkkzb1l4WEVEZERmK1BYc1piVWhNTHhKR0JBSmZmVGRLQllLTQpsN0VlTHp6enc3U2NNR3pCU2xwektpZ1dSN0VxcnJ3VHl3bU9Gd2IzenFSVEtVcXVUYzlRWVVvZVdHcHVjTjRtCmtCV3ZlQ3FYTkhtV0FncXFLUVFadkxzT0NBNXV3eVZVbW85NkNmSkprd1pRTmpPNnpkNHBFUHVhTmE2K0psWSsKeDVNM05PcW1vOXZmb3MvSWhYVmNycm9SRllKRjVhY0FMdnRCamR4N3pSZ3lFOTJvMi9tY2pHRUlLNG8yUjRNQwpBd0VBQVRBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQWdFQVZUT0RIRmIwdUlSWGtnK0MwdHNpaXJhN25GM29GOTlaCkpJdkd2dGFEcGxUeHdXdEoyenpqNSt1UzlSOE4rZlVrTE1PVW1TbjhsMkphbExwaTNXakZRWnhodkNaeVFUaEsKaGdySVNFUTV1cTFORWovUnlndWxwYWRuU0I4S0lNU2N5T2RzMDF2dWNrL00xcG9TekJBVk9NUlFYQ3kyZVFzQQpzWkhJUTFUblY3OWFWY1JlTEZZK0RxUkFScXM1c2pUUVZrbzlvYUJGdC92NlVDT3RIR3Z6aEYwamZJNHNSSENtCkx0N1lBNHRSV21OQ2F4OTgxMmV1YzVISk1QUTd2cDJMRGZXRUpYQTRqajMxQnZmR0NJTjl1MkhucjRXVUV3dkEKKzBNbFo5WEg3Uy8wejRtSDkwU2x5eWFqR3BRU3IxNHh5TGRpZFBBeGhZSmM4ei9JbHM3cTF3emlId010YUQxMAp0bEM4R2VLREtTbkkrRHFncWRRUWpveG5uOURYY1M2UUVBRzI2WDUxNktUQ1kyOEFvUHRwM2lpbnhndW04NWN3CndRNTFkekVWa2REaWdGQzFKUXNucUtxK2RoSVMzUExwYWM5R2REZUlpZUIzTlRGQkVzMTV6MExxdEdLUlNMWCsKTFFEZk5pck9pc0hMNDFEZ3hwVFo2SlkzZm5zNTl6WURBVlJUUTFpcUI3Q2s0QTlKNGEybDZ5Y2wzYXZDSDRpZApGNVZZcXBHNVZlTGZOZXBVcks1RS9NRGZLYXVvYUpob2RpaTA1K0JFS3hjY0RBcVVnSmt4Sm8rOGcxNnorMjV3CjFQYllnVkdaQzB0cDVBY2lVdGkvWUtnV0h6NXRHZDI1OFFpeW9qdTVFMUQ5ZzJzNjJpUVlFZjVzZlFMN2xWaloKMVBmTUU5NURtWjA9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K",
"rejectInvalid": "true"
}
}
}
}
}
]
}
]
}
The caCertificates property must contain a string with the CA Certificates, the client certificates have to be signed with. Format: Take the CA certificates ("-----BEGIN CERTIFICATE-----..."), concat them, encode everything base64.
The rejectInvalid property will control, wether a request will pass through the reverse proxy in case no or invalid client certificates are served in the request.
Request, that passed the authentication, will be annotated by the reverse proxy with headers prefixed by "X-SSL" which contain details on the client's certificate, like the common name.
Volumes🔗
SFH Apps can persist and share data with volumes. Volumes are declared per SFH app. If they persist, the data is stored on the HDD of the SFH.
Example:
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "ubuntu",
"tag": "latest",
"name": "ubuntucontainer",
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi"
}
},
"volumeMounts": [
{
"mountPath": "/tmp",
"subPath": "test",
"name": "testvolume"
}
]
}
],
"volumes": [
{
"name": "testvolume",
"capacity": "1Gi",
"type": "persistent"
}
]
}
A volume can't be shared between SFH Apps. If the volume should not be persistent but only be used to share data between containers in an app, use the type emptyDir.
Environment, Arguments & Command🔗
Containers can be configured by defining environment variables with the environment property, additional arguments for the process with the args property or by defining the process executed in the container with the commandproperty
All three properties are directly translated to the kubernetes equivalent. especially for arguments and commands, the behavior might differ to docker-compose.
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "ubuntu",
"tag": "latest",
"name": "ubuntucontainer",
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi"
}
},
"environment": [
{
"name": "ENABLE_DEBUG_MODE",
"value": "false"
}
],
"command": ["/bin/sh"],
"args": ["-c", "while true; do echo 'hello world'; sleep 10;done"]
}
]
}
Values - or how to configure an app🔗
An app can define values. This is a feature from helm. Values can be used in environments, args and command.
These values are predefined dynamically for an SFH:
sfhId: The sfhId of this SFH. E.g.42sponsorId: The id of the SFH sponsor. E.g.1sponsorRef: The sponsors reference of this SFH. E.g.Z0001Z0001
Values can be used with the template syntax of helm:
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "ubuntu",
"tag": "latest",
"name": "ubuntucontainer",
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi"
}
},
"environment": [
{
"name": "SPONSORID",
"value": "{{ .Values.sponsorId }}"
},
{
"name": "SPONSORREF",
"value": "{{ .Values.sponsorRef }}"
},
{
"name": "FKA_EQUINO",
"value": "{{ .Values.sponsorRef }}"
},
{
"name": "SFHID",
"value": "{{ .Values.sfhId }}"
}
]
}
]
}
Custom values can also be defined in an app and accessed via .Values.app:
{
"definitionVersion": "1.0.0",
"containers": [
{
"image": "ubuntu",
"tag": "latest",
"name": "ubuntucontainer",
"resources": {
"requests": {
"memory": "500Mi",
"cpu": "10m"
},
"limits": {
"memory": "500Mi"
}
},
"environment": [
{
"name": "DEBUG",
"value": "{{ .Values.app.debug }}"
}
]
}
],
"values": [
{
"name": "debug",
"value": "false"
}
]
}
The values will normally have the value defined in the values property. The values can be overriden for a single deployment (currently only via the api, not via the UI).
Probes🔗
Liveness, Readiness and StartupProbes can be defined via the AppDefinition. The functionality is exactly the same as documented in kubernetes. Therefore please go to the official doucmentation for further details: Kubernetes Probe Documentation
Metrics🔗
The SFH ecosystem provides the functionality to scrape metrics endpoints, that offer metrics in the prometheus format. The scraped metrics can be obtained by subscribing it at the management platform.
To enable metrics scraping, the appdefinition needs two settings. The global property metrics must be defined true. For every container port, which offers metrics and should be scraped, the metricsPath property must be set. The metricsPath must start with a / or can also be an empty string.
{
"definitionVersion": "1.0.0",
"metrics": true,
"containers": [
{
"image": "whoami",
"tag": "latest",
"name": "whoami",
"resources": {
"requests": {
"memory": "128M",
"cpu": "10m"
},
"limits": {
"memory": "128M"
}
},
"ports": [
{
"name": "http",
"containerPort": 80,
"metricsPath": "/metrics"
}
]
}
]
}
Live custom metrics can be streamed directly by subscribing a nats client to the topic sponsor.*.sfh.*.provider.*.app.<APPID>.metrics.custom of the nats server at nats.smartfactoryhost.com. To Authenticate, please use the user openid and your access-token as password, which you can get from your user profile.
nats --server nats.smartfactoryhost.com:4222 --user openid --password <TOKEN> subscribe "sponsor.*.sfh.*.provider.*.app.<APPID>.metrics.custom"
⚠️ Be aware that the token will become invalid after some minutes. So if you loose the connection and try to reconnect, get a fresh token first.
Please also have a look into the repository logs-and-metrics-example that shows you how you could stream, persist and explore your apps custom metrics using telegraf, influx and grafana.
Workloadattestation🔗
SFH App2App communcation can be secured. For more details, go to Workloadattestation.
AppDefinition JSON schema🔗
Can be found here Appdefinition Version 1