Happened with custom RTE configurations.
Solution was to include the TYPO3 ckeditor RTE plugins:
+++ public/typo3conf/ext/myext/Configuration/RTE/Article.yaml +++ public/typo3conf/ext/myext/Configuration/RTE/Article.yaml @@ -21,3 +21,4 @@ editor: imports: - resource: "EXT:rte_ckeditor/Configuration/RTE/Processing.yaml" - resource: "EXT:rte_ckeditor/Configuration/RTE/Editor/Base.yaml" + - resource: "EXT:rte_ckeditor/Configuration/RTE/Editor/Plugins.yaml"