什么是metadata

metadata就是元数据信息，他提供了基于匹配的 listeners, filter chains, routes and endpoints的额外的输入参数到过滤器，他是一种map的格式，通常是filter的名字（反向dns格式）。过滤器元数据的键值对在请求处理和连接发生时会别合并，后面的值会覆盖前面的值。元数据有一个名称空间的概念，然后是键值对。比如提供额外数据给httpConnectionManager的元数据名称空间 envoy.http_connection_manager.access_log 。另一个例子是每个service使用的cluster的元数据信息，他可能被多个过滤器使用。对于负载均衡来说，元数据提供了一种方法，来子集端点信息。关联元数据的endpoint，路由一个特定的元数据来选择端点。元数据有四种类型，分别是request类型，route类型，cluster类型，host类型。

metadata有什么作用

tracing customtag值得来源，路由元数据匹配，负载均衡子集决策，ratelimit 动作配置，基于元数据的权限控制，本地响应映射元数据过滤，等。

metadata数据来源

`envoy.filters.http.ext_authz`

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，会产生动态元数据信息。

当使用http授权服务器时，当来自授权服务器的响应头匹配 dynamic_metadata_from_headers 的配置值会产生元数据信息。动态元数据的key是匹配的头，动态元数据的值是匹配头的值。

`envoy.filters.network.ext_authz`

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，会产生动态元数据信息。

`envoy.filters.http.header_to_metadata`

配置一些规则，每条规则有header或cookie，当配置的值存在或不存在时就会触发规则，用来设置动态元数据。

比如：

http_filters:
  - name: envoy.filters.http.header_to_metadata
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config
      request_rules:
        - header: x-version
          on_header_present:
            metadata_namespace: envoy.lb
            key: version
            type: STRING
          on_header_missing:
            metadata_namespace: envoy.lb
            key: default
            value: 'true'
            type: STRING
          remove: false

上面规则显示，当x-version头存在时设置envoy.lb名称空间的key为version元数据为x-version的值，当x-version不存在时，设置envoy.lb的名称空间的key为default的元数据的值为true。

`envoy.filters.http.jwt_authn`

可以配置 payloadinmetadata ，成功验证jwt payload会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn:
  my_payload:
    iss: https://example.com
    sub: test@example.com
    aud: https://example.com
    exp: 1501281058

可以配置 headerinmetadata ，成功验证的头会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn:
  my_header:
    alg: JWT
    kid: EF71iSaosbC5C4tC6Syq1Gm647M
    alg: PS256

`envoy.filters.network.mongo_proxy`

当 emit_dynamic_metadata 为true时会产生metadata，格式如下

Name	Type	Description
key	string	The resource name in db.collection format.
value	array	A list of strings representing the operations executed on the resource (insert/update/query/delete).

`envoy.filters.network.mysql_proxy`

当发送到服务端的sql被解析后，会产生动态元数据信息，格式如下：

Name	Type	Description
`<table.db>`	string	The resource name in table.db format. The resource name defaults to the table being accessed if the database cannot be inferred.
[]	list	A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

`envoy.filters.network.postgres_proxy`

语句被解析后，会产生动态元数据，格式如下：

Name	Type	Description
`<table.db>`	string	The resource name in table.db format.
[]	list	A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

`envoy.filters.http.rbac`

会产生如下元数据：

shadow_effective_policy_id	string	The effective shadow policy ID matching the action (if any).
shadow_engine_result	string	The engine result for the shadow rules (i.e. either allowed or denied).
access_log_hint	boolean	Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

`envoy.filters.network.rbac`

会产生如下元数据：

Name	Type	Description
shadow_effective_policy_id	string	The effective shadow policy ID matching the action (if any).
shadow_engine_result	string	The engine result for the shadow rules (i.e. either allowed or denied).
access_log_hint	boolean	Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

`envoy.filters.network.zookeeper_proxy`

当每个消息被解析后，会产生如下元数据：

Name	Type	Description
string	The path associated with the request, response or event
string	The opname for the request, response or event
`<create_type>`	string	The string representation of the flags applied to the znode
string	The size of the request message in bytes
string	True if a watch is being set, false otherwise
string	The version parameter, if any, given with the request
string	The timeout parameter in a connect response
`<protocol_version>`	string	The protocol version in a connect response
string	The readonly flag in a connect response
string	The zxid field in a response header
string	The error field in a response header
`<client_state>`	string	The state field in a watch event
`<event_type>`	string	The event type in a a watch event

`envoy.filters.http.ratelimit`

当ratelimit服务返回 RateLimitResponse 带有dynamic_metadata时，会产生元数据信息。

4metadata怎么使用
4.1type.tracing.v3.CustomTag

{
  "tag": "...",
  "literal": "{...}",
  "environment": "{...}",
  "request_header": "{...}",
  "metadata": "{...}"
}

metadata：自定义tag，值从metadata中获取

案例：

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpconnectionmanager
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        portNumber: 8080
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
              name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  name: test
                  virtual_hosts:
                  - name: test
                    domains:
                    - "*"
                    routes:
                    - name: testroute
                      match: 
                        prefix: /
                      direct_response:
                        status: 200
                        body: 
                          inline_string: "prefix"
                      tracing:
                          customTags:
                          - metadata:
                              kind:
                                request: {}
                              metadataKey:
                                key: envoy.filters.http.rbac
                                path:
                                - key: istio_dry_run_allow_shadow_effective_policy_id
                            tag: istio.authorization.dry_run.allow_policy.name
                          - metadata:
                              kind:
                                request: {}
                              metadataKey:
                                key: envoy.filters.http.rbac
                                path:
                                - key: istio_dry_run_allow_shadow_engine_result
                            tag: istio.authorization.dry_run.allow_policy.result
                          - metadata:
                              kind:
                                request: {}
                              metadataKey:
                                key: envoy.filters.http.rbac
                                path:
                                - key: istio_dry_run_deny_shadow_effective_policy_id
                            tag: istio.authorization.dry_run.deny_policy.name
                          - metadata:
                              kind:
                                request: {}
                              metadataKey:
                                key: envoy.filters.http.rbac
                                path:
                                - key: istio_dry_run_deny_shadow_engine_result
                            tag: istio.authorization.dry_run.deny_policy.result
                          - literal:
                              value: latest
                            tag: istio.canonical_revision
                          - literal:
                              value: istio-ingressgateway
                            tag: istio.canonical_service
                          - literal:
                              value: mesh1
                            tag: istio.mesh_id
                          - literal:
                              value: istio-system
                            tag: istio.namespace
                          overallSampling:
                            numerator: 100
                            denominator: HUNDRED
                          randomSampling:
                            numerator: 1
                            denominator: HUNDRED
                          clientSampling:
                            numerator: 100
                            denominator: HUNDRED

路由设置metadata

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpconnectionmanager
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        portNumber: 8080
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
              name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  name: test
                  virtual_hosts:
                  - name: test
                    domains:
                    - "*"
                    routes:
                    - name: testroute
                      match: 
                        prefix: /
                      metadata:
                        filter_metadata:
                          "envoy.lb": 
                            canary: true
                      direct_response:
                        status: 200
                        body: 
                          inline_string: "prefix"

local_reply_config

{
  "mappers": [],
  "body_format": "{...}"
}

mappers:

{
  "filter": "{...}",
  "status_code": "{...}",
  "body": "{...}",
  "body_format_override": "{...}",
  "headers_to_add": []
}

filter:

{
  "status_code_filter": "{...}",
  "duration_filter": "{...}",
  "not_health_check_filter": "{...}",
  "traceable_filter": "{...}",
  "runtime_filter": "{...}",
  "and_filter": "{...}",
  "or_filter": "{...}",
  "header_filter": "{...}",
  "response_flag_filter": "{...}",
  "grpc_status_filter": "{...}",
  "extension_filter": "{...}",
  "metadata_filter": "{...}"
}

metadata_filter：

{
  "matcher": "{...}",匹配条件
  "match_if_key_not_found": "{...}"key不存在时是否匹配
}

matcher:

{
  "filter": "...",过滤名称
  "path": [],metadata路径
  "value": "{...}",匹配值
  "invert": "..."反向匹配
}

value:

{
  "null_match": "{...}",null匹配
  "double_match": "{...}",double匹配
  "string_match": "{...}",string匹配
  "bool_match": "...",bool匹配
  "present_match": "...",存在性匹配
  "list_match": "{...}"列表匹配
}

string_match:

{
  "exact": "...",
  "prefix": "...",
  "suffix": "...",
  "safe_regex": "{...}",
  "contains": "...",
  "ignore_case": "..."
}

body_format:

{
  "text_format": "...",
  "json_format": "{...}",
  "text_format_source": "{...}",
  "omit_empty_values": "...",
  "content_type": "...",
  "formatters": []
}

ef-local_reply_config.yaml

kubectl apply -f ef-local_reply_config.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpconnectionmanager
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        portNumber: 8080
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
              name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  name: test
                  virtual_hosts:
                  - name: test
                    domains:
                    - "*"
                    routes:
                    - name: testroute
                      match: 
                        prefix: /product
                      route:
                        weighted_clusters:
                          clusters:
                          - name: outbound|9080||productpage.istio.svc.cluster.local
                            weight: 100
                          total_weight: 100
                          runtime_key_prefix: test
                local_reply_config:
                  mappers:
                  - status_code: 200
                    filter:
                      metadata_filter:
                        matcher:
                          filter: envoy.lb
                          path: 
                          - key: canary
                          value:
                            string_match:
                              exact: "true"
                          invert: false
                        match_if_key_not_found: true
                    body: 
                      inline_string: "test"
                    body_format_override:
                      text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n" 
                    headers_to_add:
                    - header:
                        key: test
                        value: test
                      append: true
                  body_format:
                    text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n"

envoy.filters.http.set_metadata

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: set
spec:
  workloadSelector:
    labels:
      app: productpage
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        portNumber: 9080
        filterChain:
          destinationPort: 9080
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"
    patch:
      operation: INSERT_BEFORE
      value:         
          name: envoy.filters.http.set_metadata
          typedConfig:
            '@type': type.googleapis.com/envoy.extensions.filters.http.set_metadata.v3.Config
            metadata_namespace: envoy.lb
            value:
              canary: "true"

route match

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpconnectionmanager
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        portNumber: 8080
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
              name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  name: test
                  virtual_hosts:
                  - name: test
                    domains:
                    - "*"
                    routes:
                    - name: testroute
                      match: 
                        path: /tEst
                        case_sensitive: false
                        dynamic_metadata:
                          filter: envoy.lb
                          path:
                          - key: canary
                          value: "true"
                          invert: false
                      direct_response:
                        status: 200
                        body: 
                          inline_string: "runtime_fraction"

基于元数据的权限控制

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: apply-to
spec:
  workloadSelector:
    labels:
      app: mysqldb
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      listener:
        portNumber: 3306
        filterChain:
          filter:
            name: "envoy.filters.network.tcp_proxy"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.network.mysql_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.mysql_proxy.v3.MySQLProxy
          stat_prefix: mysql
  - applyTo: NETWORK_FILTER
    match:
      listener:
        portNumber: 3306
        filterChain:
          filter:
            name: "envoy.filters.network.tcp_proxy"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.network.rbac
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC
          stat_prefix: rbac
          rules:
             action: DENY
             policies:
               "product-viewer":
                 permissions:
                 - metadata:
                     filter: envoy.filters.network.mysql_proxy
                     path:
                     - key: t1.test
                     value:
                       list_match:
                         one_of:
                           string_match:
                             prefix: update
                 principals:
                 - any: true
          enforcement_type: CONTINUOUS

负载均衡决策

略，等讲cluster再看

header-To-Metadata Filter


{
  "request_rules": [],
  "response_rules": []
}

request_rules:

{
  "header": "...",
  "cookie": "...",
  "on_header_present": "{...}",
  "on_header_missing": "{...}",
  "remove": "..."
}

response_rules:

{
  "header": "...",
  "cookie": "...",
  "on_header_present": "{...}",
  "on_header_missing": "{...}",
  "remove": "..."
}

on_header_present,on_header_missing:

{
  "metadata_namespace": "...",
  "key": "...",
  "value": "...",
  "regex_value_rewrite": "{...}",
  "type": "...",
  "encode": "..."
}

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: set
spec:
  workloadSelector:
    labels:
      app: productpage
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        portNumber: 9080
        filterChain:
          destinationPort: 9080
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"
    patch:
      operation: INSERT_BEFORE
      value:         
          name: envoy.filters.http.header_to_metadata
          typedConfig:
            '@type': type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config
            request_rules:
            - header: x-version
              on_header_present:
                metadata_namespace: envoy.lb
                key: version
                type: STRING
              on_header_missing:
                metadata_namespace: envoy.lb
                key: default
                value: 'true'
                type: STRING
              remove: false
            response_rules:
            - header: x-version
              on_header_present:
                metadata_namespace: envoy.lb
                key: version
                type: STRING
              on_header_missing:
                metadata_namespace: envoy.lb
                key: default
                value: 'true'
                type: STRING
              remove: false

ratelimit actions

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpconnectionmanager
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        portNumber: 8080
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
              name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  name: test
                  virtual_hosts:
                  - name: test
                    domains:
                    - "*"
                    routes:
                    - name: testroute
                      match: 
                        prefix: /product
                      route:
                        rate_limits:
                        - stage: 0
                          disable_key: test
                          actions:
                          - metadata: 
                              descriptor_key: test
                              default_value: test
                              metadata_key:
                                key: envoy.lb
                                path:
                                - key: canary
                              source: DYNAMIC
                          limit:
                            dynamic_metadata:
                              metadata_key:
                                key: envoy.lb
                                path:
                                - key: canary
                        weighted_clusters:
                          clusters:
                          - name: outbound|9080||productpage.istio.svc.cluster.local
                            weight: 100
                          total_weight: 100
                          runtime_key_prefix: test

metadata-巧妙使用动态元数据

什么是metadata

metadata有什么作用

metadata数据来源

`envoy.filters.http.ext_authz`

`envoy.filters.network.ext_authz`

`envoy.filters.http.header_to_metadata`

`envoy.filters.http.jwt_authn`

`envoy.filters.network.mongo_proxy`

`envoy.filters.network.mysql_proxy`

`envoy.filters.network.postgres_proxy`

`envoy.filters.http.rbac`

`envoy.filters.network.rbac`

`envoy.filters.network.zookeeper_proxy`

`envoy.filters.http.ratelimit`

路由设置metadata

local_reply_config

envoy.filters.http.set_metadata

route match

基于元数据的权限控制

负载均衡决策

header-To-Metadata Filter

ratelimit actions

陈桂林

相关推荐

大佬们的评论抢沙发

女生也可以快速建出专业的网站

热门专题

分类目录

猜你喜欢

全新“一站式”建站，高质量、高售后的一条龙服务

微信抖音支付宝百度头条快手全平台打通信息流

觉得文章有用就打赏一下文章作者

非常感谢你的打赏，我们将继续给力更多优质内容，让我们一起创建更加美好的网络世界！

支付宝扫一扫打赏

微信扫一扫打赏

切换注册登录

切换登录注册

什么是metadata

metadata有什么作用

metadata数据来源

envoy.filters.http.ext_authz

envoy.filters.network.ext_authz

envoy.filters.http.header_to_metadata

envoy.filters.http.jwt_authn

envoy.filters.network.mongo_proxy

envoy.filters.network.mysql_proxy

envoy.filters.network.postgres_proxy

envoy.filters.http.rbac

envoy.filters.network.rbac

envoy.filters.network.zookeeper_proxy

envoy.filters.http.ratelimit

路由设置metadata

local_reply_config

envoy.filters.http.set_metadata

route match

基于元数据的权限控制

负载均衡决策

header-To-Metadata Filter

ratelimit actions

陈桂林

相关推荐

大佬们的评论 抢沙发

女生也可以快速建出专业的网站

热门专题

分类目录

猜你喜欢

全新“一站式”建站，高质量、高售后的一条龙服务

微信 抖音 支付宝 百度 头条 快手全平台打通信息流

觉得文章有用就打赏一下文章作者

非常感谢你的打赏，我们将继续给力更多优质内容，让我们一起创建更加美好的网络世界！

支付宝扫一扫打赏

微信扫一扫打赏

切换注册登录

切换登录注册

`envoy.filters.http.ext_authz`

`envoy.filters.network.ext_authz`

`envoy.filters.http.header_to_metadata`

`envoy.filters.http.jwt_authn`

`envoy.filters.network.mongo_proxy`

`envoy.filters.network.mysql_proxy`

`envoy.filters.network.postgres_proxy`

`envoy.filters.http.rbac`

`envoy.filters.network.rbac`

`envoy.filters.network.zookeeper_proxy`

`envoy.filters.http.ratelimit`

大佬们的评论抢沙发

微信抖音支付宝百度头条快手全平台打通信息流