Innovations in Time Related Expression Recognition Using LSTM Networks
Keywords:
Knowledge Distillation, Model Compression, Neural Networks, Soft LabelsAbstract
The proposed architecture leverages the strengths of both Convolutional Neural Network (CNN) and Bidirectional Long Short-Term (BLSTM) to create a robust model for temporal expression recognition in clinical texts. The CNN component effectively captures morphological and orthographic features at the character level, which enriches the semantic understanding of complex medical terminologies that are often abbreviated or have unique suffixes and prefixes. The BLSTM component excels in capturing long-range dependencies in text, which is crucial for understanding the context in which temporal expressions occur. By integrating these models with a CRF layer, the system not only predicts discrete labels but also ensures that the sequence of predicted labels is coherent and contextually appropriate, addressing the limitations of models that predict labels independently. The integration of pre-trained biomedical word vectors provides significant contextual grounding tailored to the medical domain, enhancing the model's ability to discern and interpret the nuances of medical language. This is crucial in clinical contexts where accurate interpretation of temporal phrases can be critical for patient management and treatment timelines. Further, experiments conducted on the dataset validate the effectiveness of the proposed model, demonstrating a notable improvement over traditional methods that rely heavily on hand-crafted features and rule-based approaches. Future work could explore the adaptability of this model to other subdomains of the medical field and its efficacy in processing multilingual texts, potentially increasing its applicability in global healthcare settings, with further refinement of the neural architecture and optimization of training strategies potentially yielding even better performance and faster processing times essential for real-time clinical decision support systems.
References
Moharasar, G., & Tu, B. H. (2016). A semi-supervised approach for temporal information extraction from clinical text. In Proceedings of IEEE RIVF International Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future (pp. 7-12). Piscataway, NJ: IEEE Press.
Wang, S., Liu, Z., & Peng, B. (2023, December). A Self-training Framework for Automated Medical Report Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 16443-16449).
Gong, Y., Qiu, H., Liu, X., Yang, Y., & Zhu, M. (2024). Research and Application of Deep Learning in Medical Image Reconstruction and Enhancement. Frontiers in Computing and Intelligent Systems, 7(3), 72-76.
Dai, W., Tao, J., Yan, X., Feng, Z., & Chen, J. (2023, November). Addressing Unintended Bias in Toxicity Detection: An LSTM and Attention-Based Approach. In 2023 5th International Conference on Artificial Intelligence and Computer Applications (ICAICA) (pp. 375-379). IEEE.
Xiao, M., Li, Y., Yan, X., Gao, M., & Wang, W. (2024). Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an example. arXiv preprint arXiv:2404.08279.
Li, M., Zhu, Z., Xu, R., Feng, Y., & Xiao, L. (2024). Research on Image Classification And Semantic Segmentation Model Based on Convolutional Neural Network. Journal of Computing and Electronic Information Management, 12(3), 94-100.
Lee, H. J., Xu, H., Wang, J., et al. (2016). UTHealth at SemEval-2016 task 12: An end-to-end system for temporal information extraction from clinical notes. In Proceedings of the 10th International Workshop on Semantic Evaluation (pp. 1292-1297).
Grouin, C., & Moriceau, V. (2016). LIMSI at SemEval-2016 task 12: Machine learning and temporal information to identify clinical events and time expressions. In Proceedings of the 10th International Workshop on Semantic Evaluation (pp. 1225-1230).
Cohan, A., Meurer, K., & Goharian, N. (2016). GUIR at SemEval-2016 task 12: Temporal information processing for clinical narratives. In Proceedings of the 10th International Workshop on Semantic Evaluation (pp. 1248-1255).
Barros, M., Lamurias, A., Figueiro, G., et al. (2016). ULISBOA at SemEval-2016 task 12: Extraction of temporal expressions, clinical events and relations using IBEnt. In Proceedings of the 10th International Workshop on Semantic Evaluation.
Xu, R., Yang, Y., Qiu, H., Liu, X., & Zhang, J. (2024). Research on Multimodal Generative Adversarial Networks in the Framework of Deep Learning. Journal of Computing and Electronic Information Management, 12(3), 84-88.
Zhao, W., Liu, X., Xu, R., Xiao, L., & Li, M. (2024). E-commerce Webpage Recommendation Scheme Base on Semantic Mining and Neural Networks. Journal of Theory and Practice of Engineering Science, 4(03), 207–215. https://doi.org/10.53469/jtpes.2024.04(03).20
Zhang, J., Xiao, L., Zhang, Y., Lai, J., & Yang, Y. (2024). Optimization and Performance Evaluation of Deep Learning Algorithm in Medical Image Processing. Frontiers in Computing and Intelligent Systems, 7(3), 67-71.
Yan, X., Wang, W., Xiao, M., Li, Y., & Gao, M. (2024). Survival Prediction Across Diverse Cancer Types Using Neural Networks. arXiv preprint arXiv:2404.08713.
Mikolov, T., Sutskever, I., Chen, K., et al. (2013, October 16). Distributed representations of words and phrases and their compositionality. arXiv. https://arxiv.org/abs/1310.4546
Li, Z., Yu, H., Xu, J., Liu, J., & Mo, Y. (2023). Stock market analysis and prediction using LSTM: A case study on technology stocks. Innovations in Applied Engineering and Technology, 1-6.
Yao, J., Wu, T., & Zhang, X. (2023). Improving depth gradient continuity in transformers: A comparative study on monocular depth estimation with cnn. arXiv preprint arXiv:2308.08333.
Lu, S., Liu, Z., Liu, T., & Zhou, W. (2023). Scaling-up medical vision-and-language representation learning with federated learning. Engineering Applications of Artificial Intelligence, 126, 107037.
Liu, Z., & Song, J. (2021, November). Comparison of Tree-based Feature Selection Algorithms on Biological Omics Dataset. In Proceedings of the 5th International Conference on Advances in Artificial Intelligence (pp. 165-169).
Wang, Q., Schindler, S. E., Chen, G., Mckay, N. S., McCullough, A., Flores, S., ... & Benzinger, T. L. (2024). Investigating White Matter Neuroinflammation in Alzheimer Disease Using Diffusion-Based Neuroinflammation Imaging. Neurology, 102(4), e208013.
Zhao, B., Cao, Z., & Wang, S. (2017). Lung vessel segmentation based on random forests. Electronics Letters, 53(4), 220-222.
Graves, A., & Schmidhuber, J. (2005). Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 18(5-6), 602-610.
Lafferty, J. D., McCallum, A., & Pereira, F. C. N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (pp. 282-289).
Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(7), 257-269.
White, J. (2020). PubMed 2.0. Medical reference services quarterly, 39(4), 382-387.
Bethard, S., Savova, G., Chen, W. T., Derczynski, L., Pustejovsky, J., & Verhagen, M. (2016, June). Semeval-2016 task 12: Clinical tempeval. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) (pp. 1052-1062).
Li, Y., Yan, X., Xiao, M., Wang, W., & Zhang, F. (2024). Investigation of Creating Accessibility Linked Data Based on Publicly Available Accessibility Datasets. In Proceedings of the 2023 13th International Conference on Communication and Network Security (pp. 77–81). Association for Computing Machinery.
Church, K. W. (2017). Word2Vec. Natural Language Engineering, 23(1), 155-162.
