|本期目录/Table of Contents|

[1]李 毓,杨雅婷*,李 晓,等.面向汉维机器翻译的神经网络语言模型[J].厦门大学学报(自然科学版),2019,58(02):189-194.[doi:10.6043/j.issn.0438-0479.201811020]
 LI Yu,YANG Yating*,LI Xiao,et al.Research on neural network language model for the Chinese-to-Uyghur machine translation[J].Journal of Xiamen University(Natural Science),2019,58(02):189-194.[doi:10.6043/j.issn.0438-0479.201811020]
点击复制

面向汉维机器翻译的神经网络语言模型(PDF/HTML)
分享到:

《厦门大学学报(自然科学版)》[ISSN:0438-0479/CN:35-1070/N]

卷:
58卷
期数:
2019年02期
页码:
189-194
栏目:
民族语言处理
出版日期:
2019-03-27

文章信息/Info

Title:
Research on neural network language model for the Chinese-to-Uyghur machine translation
文章编号:
0438-0479(2019)02-0189-06
作者:
李 毓12杨雅婷1*李 晓1米成刚1董 瑞1
1.中国科学院新疆理化技术研究所,新疆民族语音语言信息处理实验室,新疆 乌鲁木齐 830011; 2.中国科学院大学计算机科学与技术学院,北京 100049
Author(s):
LI Yu12YANG Yating1*LI Xiao1MI Chenggang1DONG Rui1
1.Xinjiang Laboratory of Minority Speech and Language Information Processing,the Xinjiang Technical Institute of Physics & Chemistry,Chinese Academy of Sciences,Urumqi 830011,China; 2.School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100049,China
关键词:
维吾尔语 机器翻译 语言模型 词向量 长短时序记忆网络
Keywords:
Uyghur machine translation language model word vector long short-term memory network
分类号:
H 085
DOI:
10.6043/j.issn.0438-0479.201811020
文献标志码:
A
摘要:
针对传统神经网络语言模型方法只关注词语之间关系或者词语内部信息而导致维吾尔语语言模型困惑度(PPL)过高的问题,提出了融入词素信息的维吾尔语神经网络语言模型.该方法在传统神经网络语言模型的基础上添加了词内结构建模层及合并层,利用双向长短时序记忆网络来捕捉词内结构信息,并与word2vec训练好的词向量相结合作为神经网络语言模型的输入; 同时还采用重构N元语法(N-gram)语言模型的方式将神经网络模型应用到汉维统计机器翻译中.实验表明该模型的PPL降低了19.93,在汉维统计机器翻译任务中机器双语互译评估(BLEU)值提升了0.28个百分点.
Abstract:
The traditional neural network language model is only concerned about the relationship of words or words internal information,resulting in the high perplexity of Uyghur language models.Hence,we propose a Uyghur neural network language model that incorporates morpheme information.We add the intra-word structure modeling layer and the merging layer on the traditional neural network language model.In this model,we use bi-directional long short-term memory networks to capture the intra-word structure information and combines it with word2vec-trained word vectors as the input of neural network language model.At the same time,we also adopt the method of reconstructing N-gram language model for applying neural network model to Chinese-to-Uyghur machine translation.Results show that the model exhibits a significant reduction in perplexity.In the Chinese-to-Uyghur machine translation task,the perplexity decreases by 19.93,and the BLEU value increases by 0.28 percentage point.

参考文献/References:

[1] 哈里旦木·阿布都克里木,刘洋,孙茂松.神经机器翻译系统在维吾尔语-汉语翻译中的性能对比[J].清华大学学报(自然科学版),2017(8):878-883.
[2] 李华,屈丹,张文林,等.结合全局词向量特征的循环神经网络语言模型[J].信号处理,2016,32(6):715-723.
[3] 米莉万·雪合来提,刘 凯,吐尔根·依布拉音.基于维吾尔语词干词缀粒度的汉维机器翻译[J].中文信息学报,2015,29(3):201-206.
[4] BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[M]∥Innovations in machine learning.Heidelberg:Springer,2006:137-186.
[5] MIKOLOV T,KARAFIáT M,BURGET L,et al.Recurrent neural network based language model[C]∥Interspeech 2010,Conference of the International Speech Communication Association.Makuhari:DataBase System and Logic Programming,2010:1045-1048.
[6] SUNDERMEYER M,SCHLüTER R,NEY H.LSTM neural networks for language modeling[C]∥Interspeech 2012,Conference of the International Speech Communication Association.Portland:DataBase System and Logic Programming,2012:601-608.
[7] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[8] LUONG T,SOCHER R,MANNING C.Better word representations with recursive neural networks for morphology[C]∥Proceedings of the Seventeenth Conference on Computational Natural Language Learning.Sofia:ACL,2013:104-113.
[9] RENSHAW D,HALL K B.Long short-term memory language models with additive morphological features for automatic speech recognition[C]∥2015 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Brisbane:IEEE,2015:5246-5250.
[10] MIKOLOV T,CHEN K,CORRADO G,et al.Word2vec[EB/OL].[2018-11-11].https:∥code.google.com/archive/p/word2vec.
[11] HASHIMOTO K,TSURUOKA Y.Adaptive joint learning of compositional and non-compositional phrase embeddings[EB/OL].[2018-11-11].https:∥arxiv.org/pdf/1603.06067.
[12] 刘群.统计机器翻译综述[J].中文信息学报,2003,17(4):2-13.
[13] 张家俊,宗成庆.神经网络语言模型在统计机器翻译中的应用[J].情报工程,2017,3(3):21-28.
[14] PASSBAN P,LIU Q,WAY A.Providing morphological information for statistical machine translation using neural networks[C]∥The Conference of the European Association for Machine Translation.Prague:the Prague Bulletin of Mathematical Linguistics,2017:271-282.
[15] YANG Y,MI C,MA B,et al.Character tagging-based word segmentation for Uyghur[M]∥Machine translation.Heidelberg:Springer,2014:61-69.
[16] KOEHN P,HOANG H,BIRCH A,et al.Moses:open source toolkit for statistical machine translation[C]∥Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics on Interactive Poster and Demonstration Sessions.Prague:ACL,2007:177-180.
[17] OCH F J.GIZA++:training of statistical translation models[EB/OL].[2018-11-11].http:∥www.isi.edu/och/GIZA++.html.[1] 哈里旦木·阿布都克里木,刘洋,孙茂松.神经机器翻译系统在维吾尔语-汉语翻译中的性能对比[J].清华大学学报(自然科学版),2017(8):878-883.
[2] 李华,屈丹,张文林,等.结合全局词向量特征的循环神经网络语言模型[J].信号处理,2016,32(6):715-723.
[3] 米莉万·雪合来提,刘 凯,吐尔根·依布拉音.基于维吾尔语词干词缀粒度的汉维机器翻译[J].中文信息学报,2015,29(3):201-206.
[4] BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[M]∥Innovations in machine learning.Heidelberg:Springer,2006:137-186.
[5] MIKOLOV T,KARAFIáT M,BURGET L,et al.Recurrent neural network based language model[C]∥Interspeech 2010,Conference of the International Speech Communication Association.Makuhari:DataBase System and Logic Programming,2010:1045-1048.
[6] SUNDERMEYER M,SCHLüTER R,NEY H.LSTM neural networks for language modeling[C]∥Interspeech 2012,Conference of the International Speech Communication Association.Portland:DataBase System and Logic Programming,2012:601-608.
[7] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[8] LUONG T,SOCHER R,MANNING C.Better word representations with recursive neural networks for morphology[C]∥Proceedings of the Seventeenth Conference on Computational Natural Language Learning.Sofia:ACL,2013:104-113.
[9] RENSHAW D,HALL K B.Long short-term memory language models with additive morphological features for automatic speech recognition[C]∥2015 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Brisbane:IEEE,2015:5246-5250.
[10] MIKOLOV T,CHEN K,CORRADO G,et al.Word2vec[EB/OL].[2018-11-11].https:∥code.google.com/archive/p/word2vec.
[11] HASHIMOTO K,TSURUOKA Y.Adaptive joint learning of compositional and non-compositional phrase embeddings[EB/OL].[2018-11-11].https:∥arxiv.org/pdf/1603.06067.
[12] 刘群.统计机器翻译综述[J].中文信息学报,2003,17(4):2-13.
[13] 张家俊,宗成庆.神经网络语言模型在统计机器翻译中的应用[J].情报工程,2017,3(3):21-28.
[14] PASSBAN P,LIU Q,WAY A.Providing morphological information for statistical machine translation using neural networks[C]∥The Conference of the European Association for Machine Translation.Prague:the Prague Bulletin of Mathematical Linguistics,2017:271-282.
[15] YANG Y,MI C,MA B,et al.Character tagging-based word segmentation for Uyghur[M]∥Machine translation.Heidelberg:Springer,2014:61-69.
[16] KOEHN P,HOANG H,BIRCH A,et al.Moses:open source toolkit for statistical machine translation[C]∥Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics on Interactive Poster and Demonstration Sessions.Prague:ACL,2007:177-180.
[17] OCH F J.GIZA++:training of statistical translation models[EB/OL].[2018-11-11].http:∥www.isi.edu/och/GIZA++.html.

备注/Memo

备注/Memo:
收稿日期:2018-11-12 录用日期:2018-12-03
基金项目:国家自然科学基金(U1703133); 新疆自治区重大科技专项(2016A03007-3); 新疆自治区高层次人才引进工程项目(Y839031201); 中国科学院“西部之光”人才培养引进计划(2017-XBQNXZ-A-005); 中国科学院青年创新促进会项目(2017472)
*通信作者:yangyt@ms.xjb.ac.cn
更新日期/Last Update: 1900-01-01