estceque.builtin – Builtin processors and pipeline parser

pydantic model estceque.builtin.AppendProcessor

Elasticsearch append processor.

See Append processor for more information.

field allow_duplicates: bool = True
field field: FieldPath [Required]
field media_type: str = 'application/json'
field value: Element | list[Element] [Required]
pydantic model estceque.builtin.BytesProcessor

Elasticsearch bytes processor.

See Bytes processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.CommunityIDProcessor

Elasticsearch Community ID processor.

See Community ID processor for more information.

field destination_ip: FieldPath = FieldPath('destination.ip')
field destination_port: FieldPath = FieldPath('destination.port')
field iana_number: FieldPath = FieldPath('network.iana_number')
field icmp_code: FieldPath = FieldPath('icmp.code')
field icmp_type: FieldPath = FieldPath('icmp.type')
field ignore_missing: bool = True
field seed: Annotated[int, Ge(0), Lt(65536)] = 0
Constraints:

  • ge = 0

  • lt = 65536

field source_ip: FieldPath = FieldPath('source.ip')
field source_port: FieldPath = FieldPath('source.port')
field target_field: FieldPath = FieldPath('network.community_id')
field transport: FieldPath = FieldPath('network.transport')
pydantic model estceque.builtin.ConvertProcessor

Elasticsearch convert processor.

See Convert processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
field type: Literal['integer', 'long', 'float', 'double', 'string', 'boolean', 'ip', 'auto'] [Required]
pydantic model estceque.builtin.CSVProcessor

Elasticsearch CSV processor.

See CSV processor for more information.

field empty_value: str = ''
field field: FieldPath [Required]
field ignore_missing: bool = False
field quote: Annotated[str, StringConstraints(min_length=1, max_length=1)] = '"'
Constraints:

  • min_length = 1

  • max_length = 1

field separator: Annotated[str, StringConstraints(min_length=1, max_length=1)] = ','
Constraints:

  • min_length = 1

  • max_length = 1

field target_fields: list[FieldPath] [Required]
field trim: bool = False
pydantic model estceque.builtin.DateProcessor

Elasticsearch date processor.

See Date processor for more information.

field field: FieldPath [Required]
field formats: list[str] [Required]
field locale: str = 'ENGLISH'
field output_format: str = "yyyy-MM-dd'T'HH:mm:ss.SSSXXX"
field target_field: FieldPath = FieldPath('@timestamp')
field timezone: str = 'UTC'
pydantic model estceque.builtin.DateIndexNameProcessor

Elasticsearch date index name processor.

See Date index name processor for more information.

field date_formats: str | list[str] = "yyyy-MM-dd'T'HH:mm:ss.SSSXX"
field date_rounding: Literal['y', 'M', 'w', 'd', 'h', 'm', 's'] [Required]
field field: FieldPath [Required]
field index_name_format: str = 'yyyy-MM-dd'
field index_name_prefix: str | None = None
field locale: str = 'ENGLISH'
field timezone: str = 'UTC'
pydantic model estceque.builtin.DissectProcessor

Elasticsearch dissect processor.

See Dissect processor for more information.

field append_separator: str = ''
field field: FieldPath [Required]
field ignore_missing: bool = False
field pattern: DissectPattern [Required]
pydantic model estceque.builtin.DotExpander

Elasticsearch dot expander processor.

See Dot expander processor for more information.

field field: FieldPath [Required]
field override: bool = False
field path: str | None = None
pydantic model estceque.builtin.DropProcessor

Elasticsearch drop processor.

See Drop processor for more information.

pydantic model estceque.builtin.FailProcessor

Elasticsearch fail processor.

See Fail processor for more information.

field message: str [Required]
pydantic model estceque.builtin.FingerprintProcessor

Elasticsearch fingerprint processor.

See Fingerprint processor for more information.

field fields: list[FieldPath] [Required]
field ignore_missing: bool = False
field method: Literal['MD5', 'SHA-1', 'SHA-256', 'SHA-512', 'MurmurHash3'] = 'SHA-1'
field salt: str | None = None
field target_field: FieldPath = FieldPath('fingerprint')
pydantic model estceque.builtin.GeoIPProcessor

Elasticsearch GeoIP processor.

See GeoIP processor for more information.

field database_file: str = 'GeoLite2-City.mmdb'
field download_database_on_pipeline_creation: bool = True
field field: FieldPath [Required]
field ignore_missing: bool = False
field properties: list[str] = ['continent_name', 'country_iso_code', 'country_name', 'region_iso_code', 'region_name', 'city_name', 'location']
field target_field: FieldPath = FieldPath('@timestamp')
pydantic model estceque.builtin.GrokProcessor

Elasticsearch grok processor.

See Grok processor for more information.

field ecs_compatibility: Literal['disabled', 'v1'] = 'disabled'
field field: FieldPath [Required]
field ignore_missing: bool = False
field pattern_definitions: dict[str, str] | None = None
field patterns: list[str] [Required]
field trace_match: bool = False
pydantic model estceque.builtin.GsubProcessor

Elasticsearch gsub processor.

See Gsub processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field pattern: str [Required]
field replacement: str [Required]
field target_field: FieldPath | None = None
pydantic model estceque.builtin.HTMLStripProcessor

Elasticsearch HTML strip processor.

See HTML strip processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.JoinProcessor

Elasticsearch join processor.

See Join processor for more information.

field field: FieldPath [Required]
field separator: str [Required]
field target_field: FieldPath | None = None
pydantic model estceque.builtin.JSONProcessor

Elasticsearch JSON processor.

See JSON processor for more information.

field add_to_root: bool = False
field add_to_root_conflict_strategy: Literal['replace', 'merge'] = 'replace'
field allow_duplicate_keys: bool = False
field field: FieldPath [Required]
field strict_json_parsing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.KVProcessor

Elasticsearch KV processor.

See KV processor for more information.

field exclude_keys: list[str] | None = None
field field: FieldPath [Required]
field field_split: re.Pattern [Required]
field ignore_missing: bool = False
field include_keys: list[str] | None = None
field prefix: str = ''
field strip_brackets: bool = False
field target_field: FieldPath | None = None
field trim_key: str = ''
field trim_value: str = ''
field value_split: re.Pattern [Required]
pydantic model estceque.builtin.LowercaseProcessor

Elasticsearch lowercase processor.

See Lowercase processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.NetworkDirectionProcessor

Elasticsearch network direction processor.

See Network direction processor for more information.

Validators:

  • _validate » all fields

field destination_ip: FieldPath = FieldPath('destination.ip')
Validated by:

  • _validate

field ignore_missing: bool = True
Validated by:

  • _validate

field internal_networks: list[str] | None = None
Validated by:

  • _validate

field internal_networks_field: FieldPath | None = None
Validated by:

  • _validate

field source_ip: FieldPath = FieldPath('source.ip')
Validated by:

  • _validate

field target_field: FieldPath = FieldPath('network.direction')
Validated by:

  • _validate

pydantic model estceque.builtin.RedactProcessor

Elasticsearch redact processor.

See Redact processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field pattern_definitions: dict[str, str] | None = None
field patterns: list[str] [Required]
field prefix: str = '<'
field suffix: str = '>'
pydantic model estceque.builtin.RegisteredDomainProcessor

Elasticsearch registered domain processor.

See Registered domain processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = True
field target_field: FieldPath | EmptyFieldPath = EmptyFieldPath()
pydantic model estceque.builtin.RemoveProcessor

Elasticsearch remove processor.

See Remove processor for more information.

Validators:

  • _validate » all fields

field field: FieldPath | list[FieldPath] | None = None
Validated by:

  • _validate

field ignore_missing: bool = False
Validated by:

  • _validate

field keep: FieldPath | list[FieldPath] | None = None
Validated by:

  • _validate

pydantic model estceque.builtin.RenameProcessor

Elasticsearch rename processor.

See Rename processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field override: bool = False
field target_field: FieldPath [Required]
pydantic model estceque.builtin.RerouteProcessor

Elasticsearch reroute processor.

See Reroute processor for more information.

field dataset: str = '{{data_stream.dataset}}'
field destination: FieldPath | None = None
field namespace: str = '{{data_stream.namespace}}'
pydantic model estceque.builtin.ScriptProcessor

Elasticsearch script processor.

See Script processor for more information.

field id: str | None = None
field lang: Literal['painless', 'expression', 'mustache'] = 'painless'
field params: dict[str, Any] | None = None
field source: str | dict | None = None
pydantic model estceque.builtin.SetProcessor

Elasticsearch set processor.

See Set processor for more information.

Validators:

  • _validate » all fields

field copy_from: str | None = None
Validated by:

  • _validate

field field: FieldPath [Required]
Validated by:

  • _validate

field ignore_empty_value: bool = False
Validated by:

  • _validate

field media_type: str = 'application/json'
Validated by:

  • _validate

field override: bool = True
Validated by:

  • _validate

field value: Element = None
Validated by:

  • _validate

pydantic model estceque.builtin.SetSecurityUserProcessor

Elasticsearch set security user processor.

See Set security user processor for more information.

field field: FieldPath [Required]
field properties: list[str] = ['username', 'roles', 'email', 'full_name', 'metadata', 'api_key', 'realm', 'authentication_type']
pydantic model estceque.builtin.SortProcessor

Elasticsearch sort processor.

See Sort processor for more information.

field field: FieldPath [Required]
field order: Literal['asc', 'desc'] [Required]
field target_field: FieldPath | None = None
pydantic model estceque.builtin.SplitProcessor

Elasticsearch split processor.

See Split processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field preserve_trailing: bool = False
field separator: re.Pattern [Required]
field target_field: FieldPath | None = None
pydantic model estceque.builtin.TrimProcessor

Elasticsearch trim processor.

See Trim processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.UppercaseProcessor

Elasticsearch uppercase processor.

See Uppercase processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.URIPartsProcessor

Elasticsearch URI parts processor.

See URI parts processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field keep_original: bool = True
field remove_if_successful: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.URLDecodeProcessor

Elasticsearch URL decode processor.

See URL decode processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field target_field: FieldPath | None = None
pydantic model estceque.builtin.UserAgentProcessor

Elasticsearch user agent processor.

See User agent processor for more information.

field field: FieldPath [Required]
field ignore_missing: bool = False
field properties: list[str] = ['name', 'major', 'minor', 'patch', 'build', 'os', 'os_name', 'os_major', 'os_minor', 'device']
field regex_file: str | None = None
field target_field: FieldPath = FieldPath('user_agent')
estceque.builtin.DEFAULT_INGEST_PIPELINE_PARSER = DEFAULT_INGEST_PIPELINE_PARSER

Default Elasticsearch ingest pipeline parser instance.

This instance defines all of the default processors available in all contexts, including on Elasticsearch and in Logstash’s elastic_integration filter.