Logging and Monitoring
Monitorclustercomponents
Monitorclustercomponents
To monitor kubernetes cluster can use
opensource:
--------------------->
1. Metrics Server --> no historical Metrics
2. Prometheus
3. Elastic Stack
Proprietory:
---------------->
4. Datadog
5. Dynatrace
bharathdasaraju@MacBook-Pro 2.Scheduling % minikube addons enable metrics-server
▪ Using image k8s.gcr.io/metrics-server/metrics-server:v0.4.2
🌟 The 'metrics-server' addon is enabled
bharathdasaraju@MacBook-Pro 2.Scheduling %
bharathdasaraju@MacBook-Pro 2.Scheduling % minikube addons enable dashboard
▪ Using image kubernetesui/dashboard:v2.1.0
▪ Using image kubernetesui/metrics-scraper:v1.0.4
💡 Some dashboard features require the metrics-server addon. To enable all features please run:
minikube addons enable metrics-server
🌟 The 'dashboard' addon is enabled
bharathdasaraju@MacBook-Pro 2.Scheduling % minikube dashboard --url
🤔 Verifying dashboard health ...
🚀 Launching proxy ...
🤔 Verifying proxy health ...
http://127.0.0.1:59522/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/
bharathdasaraju@MacBook-Pro 2.Scheduling % kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
minikube 493m 24% 898Mi 45%
bharathdasaraju@MacBook-Pro 2.Scheduling %
Monitorclustercomponentslabs
Monitorclustercomponentslabs
root@controlplane:~# kubectl get pods
NAME READY STATUS RESTARTS AGE
elephant 1/1 Running 0 47s
lion 1/1 Running 0 47s
rabbit 1/1 Running 0 47s
root@controlplane:~#
Deploy a metrics-server
git clone https://github.com/kodekloudhub/kubernetes-metrics-server.git
root@controlplane:~# git clone https://github.com/kodekloudhub/kubernetes-metrics-server.git
Cloning into 'kubernetes-metrics-server'...
remote: Enumerating objects: 24, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 24 (delta 4), reused 0 (delta 0), pack-reused 12
Unpacking objects: 100% (24/24), done.
root@controlplane:~#
root@controlplane:~# cd kubernetes-metrics-server/
root@controlplane:~/kubernetes-metrics-server# ls -rtlh
total 32K
-rw-r--r-- 1 root root 384 Sep 20 02:53 aggregated-metrics-reader.yaml
-rw-r--r-- 1 root root 219 Sep 20 02:53 README.md
-rw-r--r-- 1 root root 612 Sep 20 02:53 resource-reader.yaml
-rw-r--r-- 1 root root 249 Sep 20 02:53 metrics-server-service.yaml
-rw-r--r-- 1 root root 976 Sep 20 02:53 metrics-server-deployment.yaml
-rw-r--r-- 1 root root 293 Sep 20 02:53 metrics-apiservice.yaml
-rw-r--r-- 1 root root 324 Sep 20 02:53 auth-reader.yaml
-rw-r--r-- 1 root root 303 Sep 20 02:53 auth-delegator.yaml
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl create -f .
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
controlplane 502m 1% 1274Mi 0%
node01 88m 0% 374Mi 0%
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl top pod -n kube-system
NAME CPU(cores) MEMORY(bytes)
coredns-74ff55c5b-lvss6 6m 18Mi
coredns-74ff55c5b-vcmhj 5m 18Mi
etcd-controlplane 35m 42Mi
kube-apiserver-controlplane 104m 306Mi
kube-controller-manager-controlplane 37m 56Mi
kube-flannel-ds-sz9vl 5m 36Mi
kube-flannel-ds-v54mw 6m 36Mi
kube-proxy-274jt 1m 30Mi
kube-proxy-7nfrj 1m 30Mi
kube-scheduler-controlplane 8m 24Mi
metrics-server-774b56d589-2wb8t 3m 19Mi
root@controlplane:~/kubernetes-metrics-server# kubectl top pod -n default
NAME CPU(cores) MEMORY(bytes)
elephant 28m 32Mi
lion 1m 18Mi
rabbit 171m 253Mi
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
controlplane 501m 1% 1174Mi 0%
node01 103m 0% 378Mi 0%
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
controlplane 500m 1% 1237Mi 0%
node01 92m 0% 380Mi 0%
root@controlplane:~/kubernetes-metrics-server#
root@controlplane:~/kubernetes-metrics-server# kubectl top pod
NAME CPU(cores) MEMORY(bytes)
elephant 30m 30Mi
lion 1m 18Mi
rabbit 156m 253Mi
root@controlplane:~/kubernetes-metrics-server#
applogs
applogs
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % docker run kodekloud/event-simulator
Unable to find image 'kodekloud/event-simulator:latest' locally
latest: Pulling from kodekloud/event-simulator
4fe2ade4980c: Pull complete
7cf6a1d62200: Pull complete
cd7a503c982e: Pull complete
Digest: sha256:1e3e9c72136bbc76c96dd98f29c04f298c3ae241c7d44e2bf70bcc209b030bf9
Status: Downloaded newer image for kodekloud/event-simulator:latest
[2021-09-20 03:03:40,440] INFO in event-simulator: USER2 logged out
[2021-09-20 03:03:41,441] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:03:42,442] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:03:54,460] INFO in event-simulator: USER2 is viewing page1
[2021-09-20 03:03:55,462] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:04:03,445] INFO in event-simulator: USER2 logged in
[2021-09-20 03:04:04,446] WARNING in event-simulator: USER7 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:04:04,448] INFO in event-simulator: USER4 is viewing page1
[2021-09-20 03:04:05,449] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:04:05,449] INFO in event-simulator: USER3 is viewing page1
^CTraceback (most recent call last):
File "event-simulator.py", line 54, in <module>
time.sleep(1)
KeyboardInterrupt
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring %
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % docker run -d kodekloud/event-simulator
8fa3501504271d3a8fc28debbff4d569ae09772fdc2565fa56aee770220b687e
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % docker logs -f 8fa
[2021-09-20 03:06:53,275] INFO in event-simulator: USER1 logged out
[2021-09-20 03:06:54,277] INFO in event-simulator: USER2 logged in
[2021-09-20 03:06:55,278] INFO in event-simulator: USER2 logged out
[2021-09-20 03:06:56,280] INFO in event-simulator: USER3 logged in
[2021-09-20 03:06:57,282] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:06:58,249] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:06:58,250] INFO in event-simulator: USER3 is viewing page3
[2021-09-20 03:06:59,251] INFO in event-simulator: USER4 is viewing page2
[2021-09-20 03:07:00,254] INFO in event-simulator: USER3 is viewing page1
[2021-09-20 03:07:01,255] WARNING in event-simulator: USER7 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:07:01,256] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:07:02,257] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:07:03,259] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:07:03,261] INFO in event-simulator: USER3 logged in
[2021-09-20 03:07:04,263] INFO in event-simulator: USER3 logged in
[2021-09-20 03:07:05,265] INFO in event-simulator: USER4 is viewing page2
[2021-09-20 03:07:06,267] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:07:07,271] INFO in event-simulator: USER3 is viewing page2
^C%
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring %
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % kubectl run logs-events --image=kodekloud/event-simulator --dry-run=client -o yaml > event-simulator.yaml
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % kubectl create -f event-simulator.yaml
pod/logs-events created
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring %
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring % kubectl logs logs-events -f
[2021-09-20 03:18:31,489] INFO in event-simulator: USER3 logged out
[2021-09-20 03:18:32,493] INFO in event-simulator: USER1 is viewing page1
[2021-09-20 03:18:33,494] INFO in event-simulator: USER1 is viewing page1
[2021-09-20 03:18:34,495] INFO in event-simulator: USER2 is viewing page1
[2021-09-20 03:18:35,495] INFO in event-simulator: USER3 logged out
[2021-09-20 03:18:36,497] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:18:36,497] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:18:37,499] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:18:38,500] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:18:39,501] WARNING in event-simulator: USER7 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:18:39,501] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:18:40,502] INFO in event-simulator: USER1 logged in
[2021-09-20 03:18:41,502] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:18:41,503] INFO in event-simulator: USER4 is viewing page2
[2021-09-20 03:18:42,503] INFO in event-simulator: USER4 is viewing page3
^C
bharathdasaraju@MacBook-Pro 3.Logging_and_Monitoring %
applogslabs
applogslabs
root@controlplane:~# kubectl logs -f webapp-2 simple-webapp
[2021-09-20 03:23:24,120] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:23:25,122] INFO in event-simulator: USER3 is viewing page2
[2021-09-20 03:23:26,124] INFO in event-simulator: USER2 is viewing page3
[2021-09-20 03:23:27,124] INFO in event-simulator: USER2 is viewing page1
[2021-09-20 03:23:28,126] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:23:29,127] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:29,128] INFO in event-simulator: USER3 logged in
[2021-09-20 03:23:30,128] INFO in event-simulator: USER3 is viewing page3
[2021-09-20 03:23:31,130] INFO in event-simulator: USER2 is viewing page3
[2021-09-20 03:23:32,132] WARNING in event-simulator: USER30 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:23:32,132] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:23:33,133] INFO in event-simulator: USER3 is viewing page1
[2021-09-20 03:23:34,134] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:34,134] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:23:35,135] INFO in event-simulator: USER3 is viewing page3
[2021-09-20 03:23:36,136] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:23:37,138] INFO in event-simulator: USER3 is viewing page3
[2021-09-20 03:23:38,139] INFO in event-simulator: USER4 is viewing page2
[2021-09-20 03:23:39,141] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:39,141] INFO in event-simulator: USER3 is viewing page1
[2021-09-20 03:23:40,142] WARNING in event-simulator: USER30 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:23:40,142] INFO in event-simulator: USER2 logged out
[2021-09-20 03:23:41,144] INFO in event-simulator: USER3 logged out
[2021-09-20 03:23:42,145] INFO in event-simulator: USER4 logged out
[2021-09-20 03:23:43,147] INFO in event-simulator: USER4 logged out
[2021-09-20 03:23:44,149] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:44,149] INFO in event-simulator: USER2 logged in
[2021-09-20 03:23:45,151] INFO in event-simulator: USER1 is viewing page2
[2021-09-20 03:23:46,152] INFO in event-simulator: USER2 is viewing page3
[2021-09-20 03:23:47,154] INFO in event-simulator: USER4 is viewing page3
[2021-09-20 03:23:48,156] WARNING in event-simulator: USER30 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:23:48,156] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:23:49,156] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:49,157] INFO in event-simulator: USER4 logged in
[2021-09-20 03:23:50,158] INFO in event-simulator: USER1 logged in
[2021-09-20 03:23:51,160] INFO in event-simulator: USER2 is viewing page3
[2021-09-20 03:23:52,161] INFO in event-simulator: USER4 logged out
[2021-09-20 03:23:53,161] INFO in event-simulator: USER4 logged in
[2021-09-20 03:23:54,162] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:54,163] INFO in event-simulator: USER4 logged out
[2021-09-20 03:23:55,165] INFO in event-simulator: USER3 is viewing page3
[2021-09-20 03:23:56,184] WARNING in event-simulator: USER30 Order failed as the item is OUT OF STOCK.
[2021-09-20 03:23:56,185] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:23:57,186] INFO in event-simulator: USER1 logged in
[2021-09-20 03:23:58,188] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:23:59,189] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:23:59,190] INFO in event-simulator: USER2 is viewing page2
[2021-09-20 03:24:00,192] INFO in event-simulator: USER1 logged out
[2021-09-20 03:24:01,193] INFO in event-simulator: USER2 logged in
[2021-09-20 03:24:02,194] INFO in event-simulator: USER1 logged out
[2021-09-20 03:24:03,196] INFO in event-simulator: USER1 is viewing page3
[2021-09-20 03:24:04,197] WARNING in event-simulator: USER5 Failed to Login as the account is locked due to MANY FAILED ATTEMPTS.
[2021-09-20 03:24:04,198] WARNING in event-simulator: USER30 Order failed as the item is OUT OF STOCK.
Monitoring pod
podeventsimulator
podeventsimulator
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: logs-events
name: logs-events
spec:
containers:
- image: kodekloud/event-simulator
name: logs-events
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
Install Metrics-Server
Metrics Server in minikube -
used below code to install metics-server in minikube
metrics-server code at metrics-server.