Many existing integrations of natural language processing (NLP) and knowledgebased systems build on ‘shallow’ NLP. Prominent examples are term-frequency based document retrieval or document topic extraction systems. Recent progress in NLP, however, has brought ‘deep’ processing features like sentential parsing and semantic dependency analysis to a highly mature level for a substantial set of common spoken languages. Specifically, ‘deep’ NLP outputs beyond conventional syntax trees allow for better interoperability with other information or knowledge management (IM/KM) components, e.g. those using semantic technologies or statistical learning approaches. In this paper, the emerging importance of interleaved knowledge and language processing in the context of cognitive computing is shown. Building blocks for an open architecture for deep NLP applications are introduced and discussed.