Deep learning for sentiment recognition on Twitter; application to financial analysis


Working environment

This Ph.D. thesis is proposed in the context of a collaboration between the LORIA public laboratory in computer science, and the company SESAMm, who is specialized in analysis of social networks for financial predictions. The Ph.D. student will be co-supervised by:

  • Christophe Cerisara, CNRS researcher (HDR) at LORIA
  • Alexandre Denis, research engineer at SESAMm

 

Objectives

Deep neural networks constitute the most promising machine learning models to address complex Natural Language Processing (NLP) tasks, especially when a large amount of observations is available, such as for sentiment analysis on Twitter. However, very few of these observations can be manually labelled, and so training a deep network with direct supervision is not an option. One of the most important work when designing such a model is thus to devise adequate approaches to be able to train the deep models without direct supervision, i.e., without manual labels for the target task. Several strategies may be investigated for this purpose, some of them are described in the next Section. The first research objective of the Ph.D. student is to study the applicability of some of these strategies for analysing sentiment of Twitter-like micro-blogs, and to design novel effective training solutions that are efficient in this context. A second objective is to design deep networks that can be trained with these weakly supervised strategies in order to predict the evolution of financial and stock market indicators.

 

Subject description

One of the key advantage of deep neural networks, as compared to most other expert systems or machine learning techniques, is their capacity to automatically infer hierarchically more and more complex and abstract structures hidden in the observations to accomplish classification, regression and prediction tasks. For instance, in order to identify persons in a photography, they automatically learn to first identify simple shapes such as lines or circles, then to combine these shapes to recognize the oval of the face or the shape of lips, and then to map some specific shapes or distances between eyes to a given person. In the NLP domain, the same kind of hierarchical abstract features may be computed from multiple Twitter messages in order to determine the sentiment/opinion expressed in a specific Tweet or in a group of Tweets about a given product, and ultimately to help predicting the future stock market evolution of financial actors. But capturing automatically these complex structures hidden in text messages is only possible when training the model on a very large amount of data. The core of the thesis shall thus consist in designing relevant deep models for the target task, and more importantly, to propose and evaluate training strategies that can cope with large quantity of data without manual labels. The first training strategies to investigate will be: (1) Designing auxiliary tasks, similar to the ones that are used to train word embeddings, for which annotations can be computed automatically [1]; (2) Transfert and distant learning, i.e. exploiting indirect annotations [2,3]; (3) Bootstrapping and co-training, i.e. relying on confidence metrics and independent models to augment an initial small set of manual labels [4]; (4) Data augmentation, which consists in generating new observations from the annotated ones: this approach is highly successful in image recognition, but is much more challenging in the NLP domain. We may however leverage our strong expertise in the team about text generation to propose the first efficient data augmentation approaches for NLP.

The deep models shall be validated on concrete use cases in the domain of financial prediction, with the objective of deploying them in SESAMm’s products.

 

Industrial context

SESAMm is selling since 2014 text analysis applications for financial prediction. Their success has been confirmed in December 2015 with a fund raising of 640k€[1] to support their development. SESAMm will provide large confidential datasets for validating the proposed models in the target application domain, as well as preprocessing methods of these corpora. The Ph.D. student shall publish the theoretical models he will propose[2] in renown conferences and journals and compare them with the state-of-the-art on other public corpora, e.g., SemEval or as in [5]. More generally, the theoretical results carried on by the Ph.D. student will be public, while the application-related implementations and achievements will belong to SESAMm. The objective of the collaboration between LORIA and SESAMm is to enable technological transfer of the researches carried on in this Ph.D. thesis into the financial analytic products of SESAMm. A confidential and intellectual property agreement will be written and signed by the involved partners at the beginning of the thesis.

 

[1]    http://www.boursier.com/actualites/news/previsions-boursieres-sesamm-vient-de-lever-640-000-euros-668753.html

[2]    After agreement by SESAMm, and at the exclusion of any sensitive confidential information

Supervisors

Christophe Cerisara (LORIA-CNRS), Alexandre Denis (SESAMm SAS, Metz)

 

How to apply

In order to prepare a PhD thesis within the Lorraine Université d’Excellence Program, the interested candidate should consult the PhD topics offered in each social and economic challenges.
These PhD thesis topics are proposed by faculty members or researchers accredited to supervise research.

Candidate application period: according to graduate school schedule (visit each topic)
Each candidate may submit an application on up to three separate research topics.

Application analysis period by each graduate school
The graduate school reviews the applicants for a doctoral contract in the relevant disciplines. They check the level of supervision for each supervisor and the situation of trained doctors. Each candidate will meet the laboratory director, supervisor and a representative from the graduate school. This interview is to identify the candidate’s motivations and suitability as a candidate for the PhD project proposed by the supervisor. A recommendation will be made to the graduate school. This will summarize the strengths and/or weaknesses of the application.

PhD grants will include monthly income for the PhD student (roughly 1700 € for research only, complement can be provided for teaching missions) and environment for research in the research unit.

Please be aware that in order to offer a variety of subjects, more positions are posted here than available funding. The LUE executive committee will make the final choice on the granted funding (up to 12 positions), based on the recommendations by the doctoral schools.