6 month with income, possibility to continue as a PhD

Master 2 internship 2019-2020 – Sparse Matrix Linear Algebra and TensorFlow for Machine Learning on new Processors

The main goal of this internship is to experiment existing linear algebra proposed by TensorFlow for sparse matrix computation and to develop new methods using different sparse matrix patterns

Sparse Matrix Linear Algebra and TensorFlow for Machine Learning on new Processors

Supervisors

Serge Petiton (serge.petiton@univ-lille.fr) and Jérôme Gurhem, Maison de la simulation/CNRS,
Saclay

Context

Exascale supercomputers, and beyond, are expected to have highly hierarchical architectures, generating multi-level programming. The different programming associated to those new architectures and new hardware will generate new difficult challenges. Several programming paradigms and languages based on workflows or graphs of tasks are candidates. Machine learning applications on such supercomputers would generate important breakthroughs with crucial societal impacts. Nevertheless, a lot of researches on several topics still have to be addressed before any such results. An important topic is how we have to adapt linear algebra methods for future supercomputers manipulating huge data for machine learning applications, depending of programming paradigms in particular. As explain recently by Y. LeCun, future neural networks would have just a small amount of neurones actives for a given step and the main problem would be distributed and parallel spares matrix linera algebra problems, and in particular very large and sparse matrix vector multiplication. TensorFlow is an “open source software library for high performance numerical computation”, introduced by Google in 2015, well-adapted for machine learning. A TensorFlow program is based on a Graph where each node is an Operation that process Tensors (containing data). TensorFlow allows to manipulate tensors and dense and sparse matrices and propose several linear algebra methods.

Description of the subject

The main goal of this internship is to experiment existing linear algebra proposed by TensorFlow for sparse matrix computation and to develop new methods using different sparse matrix patterns. In particular, the student will analyse how it is possible to use new sparse matrix compression formats using TensorFlow in order to minimize communications and optimize computing time, on processors without any shared memories connecting all the cores. Some experiments on new processors or simulators will be proposed and analysed. We’ll probably experiment with the new parallel processor proposed by the Silicon Valley star-up Cerebras and/or the new Fujitsu multi-core chip or simulator with a network on chip, developed for the future Japanese exascale machine..

Expected contributions

  • Evaluation of TensorFlow for sparse matrix computation, with the existing
    COO format
  • Implementation and evaluation of several other sparse formats (ELLPACk,
    SGP, CRS, …) into TensorFlow, with experiments on multi-core processors.
  • Implementation of some linear algebra methods using those sparse formats
    and TensoFlow
  • Depending of the obtained results, implementation and experiments using
    several multi-cores processors in parallel.
  • Co-author of a paper to be submitted to an international conference with peer
    reviews
  • Internship report in English

Possibility to continue as a PhD

Serge Petiton - Contacter
Maison de la Simulation USR 3441
Bâtiment 565 – Digiteo
CEA Saclay
91191 Gif-sur-Yvette cedex

Autres offres d'emploi

Retour à la liste