基于OpenTelemetry+Grafana可观测性实践

axing
2025-06-20 / 0 评论 / 1 阅读 / 正在检测是否收录...

一、方案介绍

OpenTelemetry +Prometheus+ Loki + Tempo + Grafana 是一套现代化、云原生的可观测性解决方案组合,涵盖 Trace(链路追踪)、Log(日志)、Metrics(指标) 三大核心维度,为微服务架构中的应用提供统一的可观测性平台。

二、组件介绍
mc3ogsib.png
三、系统架构

mc3oh3pe.png
四、部署示例应用
4.1应用介绍

https://opentelemetry.io/docs/demo/kubernetes-deployment/

官方为大家写了一个 opentelemetry-demo。

这个项目模拟了一个微服务版本的电子商城,主要包含了以下一些项目:

mc3oicq0.png
4.2部署应用
4.2.1获取 charts 包

# helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
# helm pull open-telemetry/opentelemetry-demo --untar
# cd opentelemetry-demo 
# ls
Chart.lock  Chart.yaml  examples  grafana-dashboards  README.md  UPGRADING.md        values.yaml
charts      ci          flagd     products            templates  values.schema.json

4.2.2自定义 charts 包,默认 charts 包集成了opentelemetry-collector、prometheus、grafana、opensearch、jaeger 组件,我们先将其禁用

# vim values.yaml
default:
  # List of environment variables applied to all components
  env:
    - name: OTEL_COLLECTOR_NAME
      value: center-collector.opentelemetry.svc
opentelemetry-collector:
  enabled: false
jaeger:
  enabled: false
prometheus:
  enabled: false
grafana:
  enabled: false
opensearch:
  enabled: false

4.2.3安装示例应用

# helm install demo . -f values.yaml
- All services are available via the Frontend proxy: http://localhost:8080
  by running these commands:
     kubectl --namespace default port-forward svc/frontend-proxy 8080:8080

  The following services are available at these paths after the frontend-proxy service is exposed with port forwarding:
  Webstore             http://localhost:8080/
  Jaeger UI            http://localhost:8080/jaeger/ui/
  Grafana              http://localhost:8080/grafana/
  Load Generator UI    http://localhost:8080/loadgen/
  Feature Flags UI     http://localhost:8080/feature/
# kubectl get pod                                 
NAME                               READY   STATUS    RESTARTS      AGE
accounting-79cdcf89df-h8nnc        1/1     Running   0             2m15s
ad-dc6768b6-lvzcq                  1/1     Running   0             2m14s
cart-65c89fcdd7-8tcwp              1/1     Running   0             2m15s
checkout-7c45459f67-xvft2          1/1     Running   0             2m13s
currency-65dd8c8f6-pxxbb           1/1     Running   0             2m15s
email-5659b8d84f-9ljr9             1/1     Running   0             2m15s
flagd-57fdd95655-xrmsk             2/2     Running   0             2m14s
fraud-detection-7db9cbbd4d-znxq6   1/1     Running   0             2m15s
frontend-6bd764b6b9-gmstv          1/1     Running   0             2m15s
frontend-proxy-56977d5ddb-cl87k    1/1     Running   0             2m15s
image-provider-54b56c68b8-gdgnv    1/1     Running   0             2m15s
kafka-976bc899f-79vd7              1/1     Running   0             2m14s
load-generator-79dd9d8d58-hcw8c    1/1     Running   0             2m15s
payment-6d9748df64-46zwt           1/1     Running   0             2m15s
product-catalog-658d99b4d4-xpczv   1/1     Running   0             2m13s
quote-5dfbb544f5-6r8gr             1/1     Running   0             2m14s
recommendation-764b6c5cf8-lnkm6    1/1     Running   0             2m14s
shipping-5f65469746-zdr2g          1/1     Running   0             2m15s
valkey-cart-85ccb5db-kr74s         1/1     Running   0             2m15s
# kubectl get svc 
NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ad                ClusterIP   10.103.72.85     <none>        8080/TCP                     2m19s
cart              ClusterIP   10.106.118.178   <none>        8080/TCP                     2m19s
checkout          ClusterIP   10.109.56.238    <none>        8080/TCP                     2m19s
currency          ClusterIP   10.96.112.137    <none>        8080/TCP                     2m19s
email             ClusterIP   10.103.214.222   <none>        8080/TCP                     2m19s
flagd             ClusterIP   10.101.48.231    <none>        8013/TCP,8016/TCP,4000/TCP   2m19s
frontend          ClusterIP   10.103.70.199    <none>        8080/TCP                     2m19s
frontend-proxy    ClusterIP   10.106.13.80     <none>        8080/TCP                     2m19s
image-provider    ClusterIP   10.109.69.146    <none>        8081/TCP                     2m19s
kafka             ClusterIP   10.104.9.210     <none>        9092/TCP,9093/TCP            2m19s
kubernetes        ClusterIP   10.96.0.1        <none>        443/TCP                      176d
load-generator    ClusterIP   10.106.97.167    <none>        8089/TCP                     2m19s
payment           ClusterIP   10.102.143.196   <none>        8080/TCP                     2m19s
product-catalog   ClusterIP   10.109.219.138   <none>        8080/TCP                     2m19s
quote             ClusterIP   10.111.139.80    <none>        8080/TCP                     2m19s
recommendation    ClusterIP   10.97.118.12     <none>        8080/TCP                     2m19s
shipping          ClusterIP   10.107.102.160   <none>        8080/TCP                     2m19s
valkey-cart       ClusterIP   10.104.34.233    <none>        6379/TCP                     2m19s

4.2.4接下来创建 ingress 资源,暴露frontend-proxy 服务 8080 端口

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: demo
spec:
  entryPoints:
  - web
  routes:
    - match: Host(`demo.cuiliangblog.cn`)
      kind: Rule
      services:
        - name: frontend-proxy
          port: 8080

4.2.5创建完 ingress 资源后添加 hosts 解析并访问验证。
mc3ol3j1.png

4.3配置 Ingress 输出

以 ingress 为例,从 Traefik v2.6 开始,Traefik 原生支持使用 OpenTelemetry 协议导出追踪数据(traces),这使你可以将 Traefik 的 tracing 数据发送到兼容 OTel 的后端 。Traefik 部署可参考文档:https://www.cuiliangblog.cn/detail/section/140101250,

access 配置参考文档:https://doc.traefik.io/traefik/observability/access-logs/#opentelemetry
# vim values.yaml 
experimental: # 实验性功能配置
  otlpLogs: true # 日志导出otlp格式
additionalArguments: # 自定义启动参数
  - "--experimental.otlpLogs=true"
  - "--accesslog.otlp=true"
  - "--accesslog.otlp.grpc=true"
  - "--accesslog.otlp.grpc.endpoint=center-collector.opentelemetry.svc:4317"
  - "--accesslog.otlp.grpc.insecure=true"
metrics: # 指标
  addInternals: true # 追踪内部流量
  otlp:
    enabled: true # 导出otlp格式
    grpc: # 使用grpc协议
      endpoint: "center-collector.opentelemetry.svc:4317" # OpenTelemetry地址
      insecure: true # 跳过证书
tracing: # 链路追踪
  addInternals: true # 追踪内部流量(如重定向)
  otlp:
    enabled: true # 导出otlp格式
    grpc: # 使用grpc协议
      endpoint: "center-collector.opentelemetry.svc:4317" # OpenTelemetry地址
      insecure: true # 跳过证书

五、部署组件

0

评论 (0)

取消