{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n# Vector KRR for Dipole Moments\nIn the previous example we had the case where the differentiation of a scalar\nvalue resulted in a vector value of interest. More often, one wishes to directly train on a\nvector valued property to predict the vector valued property.\n\nThis example demonstrates vector-valued KRR for predicting electronic dipole\nmoments directly as 3D vectors. We compare the use of an isotropic coupling kernel,\nwhich essentially boils down to using individual KRR for each component of the dipole moment\nand the use a output covariance matrix that is learned during training. Readers will note\nthat this latter approach resembles the multi-task GPR.\n\nWith the isotropic kernel K(x,x') = k(x,x') I_d, the model is equivalent to\nthree independent scalar KRRs. The coupled formulation K(x,x') = k(x,x') T\nlearns a 3\u00d73 positive semi definite output covariance matrix that captures cross-component\ncorrelations, adding 6 free parameters that are learned during training.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import numpy as np\nimport matplotlib.pyplot as plt\n\nfrom mfml_qc.datasets import load_nitrophenol_data\nfrom mfml_qc.vector_krr import VectorKRR, CoupledVectorKRR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load and Split Data\nWe load the QeMFi nitrophenol dataset and extract the highest-fidelity\nelectronic dipole moments. A random 512/3000 train/test split is used.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "dataset = load_nitrophenol_data()\nX_CM = dataset[\"X_CM\"]\nDPe_hf = dataset[\"DPe\"][:, -1, :]\n\nN_TRAIN = 512 # at the highest fidelity\nN_TEST = 3000\n\nrng = np.random.RandomState(42)\nperm = rng.permutation(len(DPe_hf))\ntrain_idx, test_idx = perm[:N_TRAIN], perm[N_TRAIN : N_TRAIN + N_TEST]\n\nX_train = X_CM[train_idx]\nX_test = X_CM[test_idx]\nY_train = DPe_hf[train_idx]\nY_test = DPe_hf[test_idx]\n\nY_mean = np.mean(Y_train, axis=0)\nY_train_c = Y_train - Y_mean" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train Both Models\nThe isotropic model uses the identity as output covariance (T = I),\nwhile the coupled model learns a full 3\u00d73 PSD matrix T = L L^T via\nNelder-Mead optimisation on a held-out validation set. This validation\nset is taken from the training data itself, here, we will use\na 15% validation size. In practice, therefore, the coupled\napproach 'sees' the same amount of data as the isotropic case.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "model_iso = VectorKRR(kernel_type=\"gaussian\", sigma=250.0, reg=1e-10)\nmodel_iso.train(X_train, Y_train_c)\npred_iso = model_iso.predict(X_test) + Y_mean\n\nmodel_coupled = CoupledVectorKRR(\n kernel_type=\"gaussian\", sigma=250.0, reg=1e-10, val_fraction=0.15, max_opt_iter=500\n)\nmodel_coupled.train(X_train, Y_train_c)\npred_coupled = model_coupled.predict(X_test) + Y_mean" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate\nWe compute the component-wise MAE, overall MAE, mean cosine similarity,\nand mean angular error for both models.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def eval_model(pred, true):\n comp_mae = np.mean(np.abs(pred - true), axis=0)\n overall = np.mean(np.abs(pred - true))\n mag_p = np.linalg.norm(pred, axis=-1)\n mag_t = np.linalg.norm(true, axis=-1)\n dot = np.sum(pred * true, axis=-1)\n denom = np.maximum(mag_p * mag_t, 1e-12)\n cos_sim = np.clip(dot / denom, -1.0, 1.0)\n return comp_mae, overall, cos_sim\n\n\ncomp_iso, mae_iso, cos_iso = eval_model(pred_iso, Y_test)\ncomp_cpl, mae_cpl, cos_cpl = eval_model(pred_coupled, Y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Comparison Figure\nA 2\u00d73 parity plot grid shows predicted vs true values for each dipole\ncomponent. The top row is the isotropic model, the bottom row the\ncoupled model. Points are coloured by dipole magnitude.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "fig, axes = plt.subplots(\n 2, 3, figsize=(15, 9), gridspec_kw={\"hspace\": 0.35, \"wspace\": 0.3}\n)\n\nmag_true = np.linalg.norm(Y_test, axis=-1)\ncmap = plt.cm.viridis\nvmin, vmax = mag_true.min(), mag_true.max()\ncomp_labels = [\"\u03bc_x\", \"\u03bc_y\", \"\u03bc_z\"]\nrows = [\n (\"Isotropic (T = I)\", pred_iso, comp_iso),\n (\"Coupled (learnable T)\", pred_coupled, comp_cpl),\n]\n\nfor row, (label, pred, comp_mae) in enumerate(rows):\n for c in range(3):\n sc = axes[row, c].scatter(\n Y_test[:, c],\n pred[:, c],\n s=2,\n alpha=0.4,\n c=mag_true,\n cmap=cmap,\n vmin=vmin,\n vmax=vmax,\n )\n lo = min(Y_test[:, c].min(), pred[:, c].min())\n hi = max(Y_test[:, c].max(), pred[:, c].max())\n axes[row, c].plot([lo, hi], [lo, hi], \"r--\", lw=1.5)\n axes[row, c].set_xlabel(f\"True {comp_labels[c]} (Debye)\")\n axes[row, c].set_ylabel(f\"Predicted {comp_labels[c]}\")\n axes[row, c].set_title(f\"{label} \u2014 {comp_labels[c]} (MAE: {comp_mae[c]:.4f})\")\n\nfig.subplots_adjust(right=0.88)\ncbar_ax = fig.add_axes([0.90, 0.15, 0.015, 0.7])\nfig.colorbar(sc, cax=cbar_ax).set_label(\"|\u03bc| (Debye)\")\n\nfig.suptitle(\n f\"Vector KRR (N={N_TRAIN}): \"\n f\"Isotropic MAE={mae_iso:.4f} vs Coupled MAE={mae_cpl:.4f}\",\n fontsize=12,\n fontweight=\"bold\",\n)\nplt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.22" } }, "nbformat": 4, "nbformat_minor": 0 }