deleting a document from Elasticsearch data stream
It sounds super simple to remove a document from an Elasticsearch data stream, isn’t it? But this is what happened this morning when I am trying to do so…
Me: Let’s run a delete like this~
DELETE log-basic-dev/_doc/7vIUq3wBugmWpr_4Z3ps
Kibana: bomber~ (lalalalala)
{
“error” : {
“root_cause” : [
{
“type” : “illegal_argument_exception”,
“reason” : “only write ops with an op_type of create are allowed in data streams”
}
],
“type” : “illegal_argument_exception”,
“reason” : “only write ops with an op_type of create are allowed in data streams”
},
“status” : 400
}
datastreams supports “create” only?
Not exactly, datastreams by default assumes creation of data. Also it has strict rules on how to “create” the data as well:
- only POST creation is accepted (i.e. no PUT operations)
- if using _bulk api, the only op_type accepted is “create” (i.e. no “index”, “update” and “delete”)
So does it mean we can’t delete stale documents? If really have to, we would need to query out which backing index containing that document first. Usually a search query could identify which backing index it belongs to:
{
“took” : 5,
“timed_out” : false,
“_shards” : {… },
“hits” : {
“hits” : [
{
“_index” : “shrink-wmox-.ds-log-basic-dev-2021.10.23–000007”,
“_type” : “_doc”,
“_id” : “URVyq3wBdZHgTDXtPN0t”,
…}
]}}
by examining the meta data from the search response, we could get back the backing index’s name and its document id.
Me: Alright~ Now let’s delete that document directly from the backing index~
DELETE shrink-wmox-.ds-log-basic-dev-2021.10.23–000007/_doc/URVyq3wBdZHgTDXtPN0t
kibana: bleh~
{
“error” : {
“root_cause” : [
{
“type” : “cluster_block_exception”,
“reason” : “index [shrink-wmox-.ds-log-basic-dev-2021.10.23–000007] blocked by: [FORBIDDEN/8/index write (api)];”
}
],
“type” : “cluster_block_exception”,
“reason” : “index [shrink-wmox-.ds-log-basic-dev-2021.10.23–000007] blocked by: [FORBIDDEN/8/index write (api)];”
},
“status” : 403
}
Now what? Forbidden/8/index write (api). Talk to me (humans here)~
The error message is really… After some diggings, I found that the targeted index has been made read-only. As “shrunk” index usually is read-only for performance reasons. Hence, all I need to do now is to revert this behaviour.
PUT shrink-wmox-.ds-log-basic-dev-2021.10.23–000007/_settings
{
“index.blocks.write”: false
}
and delete again…
DELETE shrink-wmox-.ds-log-basic-dev-2021.10.23–000007/_doc/URVyq3wBdZHgTDXtPN0t
kibana: congratulations~
{
“_index” : “shrink-wmox-.ds-log-basic-dev-2021.10.23–000007”,
“_type” : “_doc”,
“_id” : “URVyq3wBdZHgTDXtPN0t”,
“_version” : 2,
“result” : “deleted”,
“_shards” : {…},
“_seq_no” : 1,
“_primary_term” : 2
}
finally~
wrap up
In this scenario, I have revert the read-only behaviour of a shrunk index and therefore able to delete given documents within a datastream. However, for logical and performance reasons, once the deletion is done, the backing index should be re-applied to read-only (set index.blocks.write to true again)
