Search

PhysChem: Deep Molecular Representation Learning via Fusing Physical and Chemical Information

Type
Technical
Published year
2021
Journal / Conference
NeurIPS
Keyword
Multi-task
Model fusion
Chemical Conformation
Molecular Property
Status
Done
Language
πŸ‡ΊπŸ‡Έ
Blog upload
Yes
Date
2022/07/29
1 more property
β€’
NeurIPS 2021 Poster
β€’
OpenReview
https://openreview.net/forum?id=Uxi7X1EqywV&noteId=WA2nWkN6AjS

Summary

β€’
Used physicist network (PhysNet) and chemist network (ChemNet) simultaneously, and each network shares information to solve individual tasks.
β€’
PhysNet: Neural physical engine. Mimics molecular dynamics to predict conformation.
β€’
ChemNet: Message passing network for chemical & biomedical property prediction.
β€’
Molecule without 3D conformation can be inferred during test time.

Preliminaries

β€’
Molecular representation learning
Embedding molecules into latent space for downstream tasks.
β€’
Neural Physical Engines
Neural networks are capable of learning annotated potentials and forces in particle systems.
HamNet proposed a neural physical engine that operated on a generalized space, where positions and momentums of atoms were defined as high-dimensional vectors.
β€’
Multi-task learning
Sharing representations for different but related tasks.
β€’
Model fusion
Merging different models on identical tasks to improve performance.

Notation

β€’
Graph M=(V,E,n,m,Xv,Xe)\mathcal{M} = (\mathcal{V}, \mathcal{E}, n, m, \mathbf{X}^v, \mathbf{X}^e)
V\mathcal{V}: set of nn atoms
E\mathcal{E}: set of mm chemical bonds
Xv∈RnΓ—dv=(x1v,...,xnv)⊀\mathbf{X}^v \in \mathbb{R}^{n \times d_v} = (x^v_1, ..., x^v_n)^\top: matrix of atomic features
Xe∈RmΓ—de=(x1e,...,xme)⊀\mathbf{X}^e \in \mathbb{R}^{m \times d_e} = (x^e_1, ..., x^e_m)^\top: matrix of bond features

Model

β€’
Initializer
β—¦
Input: atomic features, bond features (from RDKit)
β—¦
Layer: fully connected layers
β—¦
Output:
β–ͺ
bond states, atom states for ChemNet
vi(0)=FC(xiv),i∈Vv^{(0)}_i = \text{FC}(x^v_i), i\in \mathcal{V}, ei,j(0)=FC(xi,je),(i,j)∈Ee^{(0)}_{i,j} = \text{FC}(x^e_{i,j}), (i, j)\in \mathcal{E}
β–ͺ
atom positions, atomic momenta for PhysNet
Bond strength adjacency matrix A(i,j)={0,ifΒ (i,j)βˆ‰EFCsigmoid(xi,je),ifΒ (i,j)∈EA(i,j)=\begin{cases}0, & \text{if $(i,j) \notin \mathcal{E}$}\\ \text{FC}_{\text{sigmoid}}(x^e_{i,j}), & \text{if $(i,j) \in \mathcal{E}$} \end{cases}
V~=GCN(A,V(0))\tilde{V} = \text{GCN}(A, V^{(0)})
{(qi(0)βŠ•pi(0))}=LSTM({v~i}),i∈V\{ (q^{(0)}_i \oplus p^{(0)}_i)\} = \text{LSTM}(\{\tilde{v}_i\}), i \in \mathcal{V}
β€’
PhysNet
β—¦
PhysNet is inspired by HamNet.
HamNet showed that neural networks can simulate molecular dynamics for conformation prediction.
β—¦
Directly parameterize the forces between each pair of atoms.
β—¦
Consider the effects of chemical interactions(e.g. bond types) by cooperating with ChemNet’s bond states.
β—¦
Introduces torsion forces.
β—¦
Output: 3D conformation
β€’
ChemNet
ChemNet modifies MPNN(message passing neural network) for molecular representation learning.
β—¦
Output: Molecule representation

Loss

β€’
LphysL_{\text{phys}}: Conn-k loss for Conformation prediction (PhysNet)
kk-hop connectivity loss
LConnβˆ’k(R^,R)=βˆ₯1nC^(k)βŠ™(D^βˆ’D)βŠ™(D^βˆ’D)βˆ₯FL_{\text{Conn}-k}(\hat{\mathbf{R}}, \mathbf{R}) = \|\frac{1}{n} \hat{\mathbf{C}}^{(k)} \odot (\hat{\mathbf{D}} - \mathbf{D}) \odot (\hat{\mathbf{D}} - \mathbf{D}) \|_{F}
βŠ™\odot: element-wise product
βˆ₯β‹…βˆ₯\| \cdot \|: Frobenius norm
(D^βˆ’D)(\hat{\mathbf{D}} - \mathbf{D}) : distance matrix of the real and predicted conformations (R^βˆ’R)(\hat{\mathbf{R}} - \mathbf{R})
C^(k)\hat{\mathbf{C}}^{(k)}: normalized kk-hop connectivity matrix
β€’
LchemL_{\text{chem}}: MAE or Cross entropy loss for Property prediction (ChemNet)
β€’
Total loss
Ltotal=Ξ»Lphys+LchemL_{\text{total}} = \lambda L_{\text{phys}} + L_{\text{chem}}
Checkpoints
β€’
Is Conn-k loss generally used in other conformation prediction models?
No! But seems related to local distance loss.
β€’
Is triplet descriptor generally used in other models?
No!