Reason was that the file name and the cliKeys script name were different.
CliRequestHandler tried to include() the file, which did not exist. This is a warning in PHP, and TYPO3 does not show warnings :/