Research Inspirations
Journal Club
In our weekly team meetings, we host a journal club where each lab member is expected to present a research paper in turn. This forum encourages critical thinking and keeps our team updated on recent advancements in the field. Below is our tentative list for the next journal club rotation. We’ll pull from these items in order; presenters can swap as needed.
| Category | Paper | Reason |
|---|---|---|
| 1. Architectural Foundations | The Annotated Transformer | A clear and accessible explanation of the Transformer architecture; easier to follow than the original paper. |
| Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau et al., 2015) | The origin of the attention mechanism, which inspired the Transformer. | |
| Identity Mappings in Deep Residual Networks (He et al., 2016) | Introduced residual connections that stabilize deep network training. | |
| 2. Scaling and Efficient Training | GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism | A key solution for large-scale distributed training. |
| Better & Faster Large Language Models via Multi-token Prediction (DeepMind, 2024) | A new-generation paradigm for efficient language modeling (predicting multiple tokens). | |
| Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Gu & Dao, 2023) | A state-space model achieving efficient long-sequence processing, offering an alternative to Transformer architectures. | |
| 3. Representation and Multimodal Learning | CLIP: Learning Transferable Visual Models from Natural Language Supervision | A groundbreaking multimodal framework that directly influenced GPT-4V and Gemini models. |
| 4. Knowledge Augmentation and Alignment | Reinforcement Learning from Human Feedback (RLHF) | A core technique for aligning LLMs with human preferences, combining SFT, reward modeling, and RL (e.g., PPO). |
| Retrieval-Augmented Generation (RAG) | Key mechanism for combining retrieval and generative models. | |
| 5. Alignment and Behavior Understanding | Zephyr: Direct Distillation of LM Alignment | Lightweight distillation method for alignment, an important step after RLHF. |
| Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023) | Simplifies alignment by directly optimizing model outputs using preference data, bypassing explicit reward modeling. | |
| Lost in the Middle: How Language Models Use Long Contexts | Empirical study showing that long-context models mainly focus on the beginning and end of input windows. |
Paper Reading List
The reading list below is tailored for new lab members, featuring a selection of the latest papers from leading journals. These publications reflect a broad spectrum of topics that are loosely connected to our main research interests. All new members are encouraged to explore these articles to stay at the forefront of emerging trends and developments in our field:
Acute Kidney Injury:
- Feng Y, Wang AY, Jun M, et al. Characterization of Risk Prediction Models for Acute Kidney Injury A Systematic Review and Meta-analysis. JAMA Network Open. 2023;6(5). doi:10.1001/jamanetworkopen.2023.13359
- Haredasht FN, Vanhoutte L, Vens C, et al. Validated risk prediction models for outcomes of acute kidney injury: a systematic review. BMC Nephrology. 2023;24(1). doi:10.1186/s12882-023-03150-0
- Hu J, Xu J, Li M, et al. Identification and validation of an explainable prediction model of acute kidney injury with prognostic implications in critically ill children: a prospective multicenter cohort study. eClinicalMedicine. 2024;68. doi:10.1016/j.eclinm.2023.102409
- Rank N, Pfahringer B, Kempfert J, et al. Deep-learning-based real-time prediction of acute kidney injury outperforms human predictive performance. npj Digital Medicine. 2020;3(1). doi:10.1038/s41746-020-00346-8
- Wu C, Zhang Y, Nie S, et al. Predicting in-hospital outcomes of patients with acute kidney injury. Nature Communications. 2023;14(1). doi:10.1038/s41467-023-39474-6
Dialysis:
- Cooper BA, Branley P, Bulfone L, et al. A Randomized, Controlled Trial of Early versus Late Initiation of Dialysis. New England Journal of Medicine. 2010;363(7):609-619. doi:10.1056/nejmoa1000552
- Hemmige V, Deshpande P, Norris KC, et al. Geographic Dialysis Facility Density and Early Dialysis Initiation. JAMA Network Open. 2024;7(1):E2350009. doi:10.1001/jamanetworkopen.2023.50009
- McCoy IE, Weinhandl E, Hussein W, Hsu CY. Initial Management and Potential Opportunities to Deprescribe Dialysis among Patients with AKI-D Patients after Hospital Discharge. Journal of the American Society of Nephrology. 2023;34(12):1949-1951. doi:10.1681/ASN.0000000000000225
- The STARRT‐AKI Investigators. Timing of Initiation of Renal-Replacement Therapy in Acute Kidney Injury. New England Journal of Medicine. 2020;383(3):240-251. doi:10.1056/NEJMoa2000741
Transfer Learning:
- Ebbehoj A, Thunbo MØ, Andersen OE, et al. Transfer learning for non-image data in clinical research: A scoping review. PLOS Digital Health. 2022;1(2):e0000014. doi:10.1371/journal.pdig.0000014
- Gao Y, Cui Y. Deep transfer learning for reducing health care disparities arising from biomedical data inequality. Nature Communications. 2020;11(1). doi:10.1038/s41467-020-18918-3
- Qi T, Wu F, Wu C, et al. Differentially private knowledge transfer for federated learning. Nature Communications. 2023;14(1). doi:10.1038/s41467-023-38794-x
- Radhakrishnan A, Ruiz Luyten M, Prasad N, et al. Transfer Learning with Kernel Methods. Nature Communications. 2023;14(1). doi:10.1038/s41467-023-41215-8
- Zhao Z, Alzubaidi L, Zhang J, et al. A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations. Expert Systems with Applications. 2024;242. doi:10.1016/j.eswa.2023.122807
Federated Learning:
- Dayan I, Roth HR, Zhong A, et al. Federated learning for predicting clinical outcomes in patients with COVID-19. Nature Medicine. 2021;27(10):1735-1743. doi:10.1038/s41591-021-01506-3
- Kaissis G, Ziller A, Passerat-Palmbach J, et al. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence. 2021;3(6):473-484. doi:10.1038/s42256-021-00337-8
- Zhang D, Tong J, Jing N, et al. Learning competing risks across multiple hospitals: one-shot distributed algorithms. Journal of the American Medical Informatics Association. 2024;31(5):1102-1112. doi:10.1093/jamia/ocae027
- Zhang D, Tong J, Stein R, et al. One-shot distributed algorithms for addressing heterogeneity in competing risks data across clinical sites. Journal of Biomedical Informatics. 2024;150. doi:10.1016/j.jbi.2024.104595
Causal Learning:
- Feuerriegel S, Frauen D, Melnychuk V, et al. Causal machine learning for predicting treatment outcomes. Nature Medicine. 2024;30(4):958-968. doi:10.1038/s41591-024-02902-1
- Madumal P, Miller T, Sonenberg L, Vetere Victoria F. Explainable Reinforcement Learning through a Causal Lens. www.aaai.org
- Prosperi M, Guo Y, Sperrin M, et al. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nature Machine Intelligence. 2020;2(7):369-375. doi:10.1038/s42256-020-0197-y