Abstract: Language is one of the main methods humans communicate with each other andexpress themselves, hence containing in itself an encoding of rich psychological traits ofthe speaker. By applying natural language processing (NLP) approach, human-generatedtext data such as social media posts are a useful source for researchers to gain a betterunderstanding of human psychology and behaviors. With the advent of transformersarchitecture which helps advanced NLP research, such improvements have cascaded toother interdisciplinary research involving language. This thesis aims to explore how wecan take advantage of this strong deep learning framework to tackle many psychologicalproblems, by modifying and adapting transformers to fit the characteristics of psychologystudies. Our approach focuses on two main strengths of transformers: capturingsemantically rich contextualized embeddings and the ability to generate human-like texts.Then, we propose how they can be applied to gain further understanding the relationshipbetween language and underlying psychological constructs.