Show simple item record

dc.contributor.advisorBreslin, John G.
dc.contributor.authorRuder, Sebastian
dc.date.accessioned2019-09-26T07:49:40Z
dc.date.available2019-09-26T07:49:40Z
dc.date.issued2019-06-07
dc.identifier.urihttp://hdl.handle.net/10379/15463
dc.description.abstractThe current generation of neural network-based natural language processing models excels at learning from large amounts of labelled data. Given these capabilities, natural language processing is increasingly applied to new tasks, new domains, and new languages. Current models, however, are sensitive to noise and adversarial examples and prone to overfitting. This brittleness, together with the cost of attention, challenges the supervised learning paradigm. Transfer learning allows us to leverage knowledge acquired from related data in order to improve performance on a target task. Implicit transfer learning in the form of pretrained word representations has been a common component in natural language processing. In this dissertation, we argue that more explicit transfer learning is key to deal with the dearth of training data and to improve downstream performance of natural language processing models. We show experimental results transferring knowledge from related domains, tasks, and languages that support this hypothesis. We make several contributions to transfer learning for natural language processing: Firstly, we propose new methods to automatically select relevant data for supervised and unsupervised domain adaptation. Secondly, we propose two novel architectures that improve sharing in multi-task learning and outperform single-task learning as well as the state-of-the-art. Thirdly, we analyze the limitations of current models for unsupervised cross-lingual transfer and propose a method to mitigate them as well as a novel latent variable cross-lingual word embedding model. Finally, we propose a framework based on fine-tuning language models for sequential transfer learning and analyze the adaptation phase.en_IE
dc.publisherNUI Galway
dc.rightsAttribution-NonCommercial-NoDerivs 3.0 Ireland
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/3.0/ie/
dc.subjectnatural language processingen_IE
dc.subjectmachine learningen_IE
dc.subjectdeep learningen_IE
dc.subjecttransfer learningen_IE
dc.subjectEngineering and Informaticsen_IE
dc.subjectInformation technologyen_IE
dc.subjectComputer scienceen_IE
dc.titleNeural transfer learning for natural language processingen_IE
dc.typeThesisen
dc.contributor.funderIrish Research Council for Science, Engineering and Technologyen_IE
dc.local.noteThis dissertation demonstrates that neural networks in natural language processing that leverage existing relevant information from related domains, tasks, and languages outperform models not using this information across a wide range of tasks and proposes new algorithms for transfer learning in these settings.en_IE
dc.local.finalYesen_IE
nui.item.downloads14380


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution-NonCommercial-NoDerivs 3.0 Ireland
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivs 3.0 Ireland