{"componentChunkName":"component---src-templates-post-tsx","path":"/machine_learning_models/lasso_regression/","result":{"data":{"logo":{"childImageSharp":{"gatsbyImageData":{"layout":"fixed","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/e43c6530b7b2b056de6bfee951c9e393/b181d/compasslogo.png","srcSet":"/static/e43c6530b7b2b056de6bfee951c9e393/b181d/compasslogo.png 1920w","sizes":"1920px"},"sources":[{"srcSet":"/static/e43c6530b7b2b056de6bfee951c9e393/e02c9/compasslogo.webp 1920w","type":"image/webp","sizes":"1920px"}]},"width":1920,"height":472}}},"mdx":{"body":"const _excluded=[\"components\"];function _extends(){_extends=Object.assign||function(target){for(var i=1;i=0)continue;if(!Object.prototype.propertyIsEnumerable.call(source,key))continue;target[key]=source[key];}}return target;}function _objectWithoutPropertiesLoose(source,excluded){if(source==null)return{};var target={};var sourceKeys=Object.keys(source);var key,i;for(i=0;i=0)continue;target[key]=source[key];}return target;}/* @jsxRuntime classic */ /* @jsx mdx */const _frontmatter={\"layout\":\"post\",\"title\":\"Lasso Regression Explained, Step by Step\",\"author\":[\"Lari Giba\"],\"tags\":[\"Machine Learning Models\"],\"image\":\"img/Lasso/LassoRegression_Thumbnail.png\",\"date\":\"2021-06-20T16:35:36.458Z\",\"dateModified\":\"2021-06-26T08:28:02.234Z\",\"changelog\":[[\"2021-06-26T08:28:02.234Z\",\"Added section about SGDRegressor.\"],[\"2022-01-18T09:43:47.766Z\",\"Fixed small typo in section about qualitative difference between Ridge and Lasso. Thanks to Tianyu for pointing this out!\"]],\"draft\":false,\"imgCaption\":\"Background image by Nick Tiemeyer\",\"imgSource\":\"https://unsplash.com/photos/i9cldzefwJY\",\"timeToRead\":16,\"uniquePostId\":10,\"excerpt\":\"Lasso regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. Lasso regression is very similar to ridge regression, but there are some key differences between the two that you will have to understand if you want to use them effectively. In this article, you will learn everything you need to know about lasso regression, the differences between lasso and ridge, as well as how you can start using lasso regression in your own machine learning projects.\",\"dependencies\":[],\"proficiencyLevel\":\"Beginner\"};const makeShortcode=name=>function MDXDefaultShortcode(props){console.warn(\"Component \"+name+\" was not imported, exported, or provided by MDXProvider as global scope\");return mdx(\"div\",props);};const ConditionalLazyPlot=makeShortcode(\"ConditionalLazyPlot\");const InfoTip=makeShortcode(\"InfoTip\");const Collapsible=makeShortcode(\"Collapsible\");const ConditionalLazyPlotInteractive=makeShortcode(\"ConditionalLazyPlotInteractive\");const WarningTip=makeShortcode(\"WarningTip\");const HelpfulTip=makeShortcode(\"HelpfulTip\");const ToolTip=makeShortcode(\"ToolTip\");const layoutProps={_frontmatter};const MDXLayout=\"wrapper\";return function MDXContent(_ref){let{components}=_ref,props=_objectWithoutProperties(_ref,_excluded);return mdx(MDXLayout,_extends({},layoutProps,props,{components:components,mdxType:\"MDXLayout\"}),mdx(\"h2\",{\"id\":\"outline\"},`Outline`),mdx(\"p\",null,`You have probably heard about linear regression. Maybe you have even read\nan `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/linear_regression\"},`article`),` about it.\nLasso and ridge regression are two of the most popular variations of\nlinear regression which try to make it a bit more `,mdx(\"em\",{parentName:\"p\"},`robust`),`. Nowadays it is actually very uncommon\nto see regular linear regression out in the wild, and not one of its variations like lasso or ridge.\nThe purpose of lasso and ridge is to\nstabilize the vanilla linear regression and make it more robust against outliers, overfitting,\nand more. Lasso and ridge are very similar, but there are also some key differences between the two\nthat you really have to understand if you want to use them confidently in practice.\nThis article is the second article in a series where we’ll take a deep dive into ridge and lasso regression!\nWe will look at how we might have come up with lasso regression ourselves,\ndistinguish it from ridge regression, and we’ll also code it up\nin Python. Let’s start!`),mdx(\"h2\",{\"id\":\"prerequisites\"},`Prerequisites`),mdx(\"p\",null,`This article is a direct follow up to the article `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression\"},`Ridge Regression Explained, Step by Step`),`,\nso ideally you should read the article about ridge regression before you read this article.\nIn the linked article, we construct ridge regression from scratch and go through all of the details\nnecessary to understand ridge. Since lasso is very similar to ridge, we’ll use a similar approach in this post.\nHowever, this article will be a lot more fast-paced since the details have already been covered in the article about ridge.\nSo I highly recommend you read that article first, before coming back to this one.\nWith that being said, let’s take a look at lasso regression!`),mdx(\"h2\",{\"id\":\"the-problem\"},`The Problem`),mdx(\"p\",null,`So what is wrong with linear regression? Why do we need more machine learning algorithms\nthat do the same thing? And why are there two of them? We’ve explored this question in the\n`,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression\"},`article about ridge regression`),`. Here’s a quick recap:`),mdx(\"p\",null,`We had a dataset of figure prices, where each entry in the dataset contained the age of the figure as well as its price for that age in € (or any other currency).\nWe then wanted to predict the price of a figure given its age using linear regression, to see how much the figures depreciate over time.\nThe dataset looked like this:`),mdx(ConditionalLazyPlot,_extends({},olsVsRidgeData,{mdxType:\"ConditionalLazyPlot\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"1106px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"63%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAABS0lEQVQ4y52Tf3OCMAyGPUEFJy2gKIityG+9nXfb9/9u7y6ZRep2N90fOULTPEmaZDKLjnBDjVXaQuY95L6H2HeYxwWfk/0ZMXcnpEyFwjKpGLhKGwRZawHp+5eYe5PF+oQgbeCnPdyogCMVy7OZ/ciQgCJrsUx7LNYlZpF+GWYBqbQw7+HIg2V0X8xyAIb5GUHajgwKXlLD33UvQQfg+vjOwPvDGmD7PyCPS9ZCqCsWm2rUuW8YPYk8ftwdKQg5hxretkGgrlZgbkp0OGMWF+z8GHnO31ujQs3Nowoem2c1RWQdH9jjoq2LFHAa5AiLT4Y6IrdmdJhDU563OSFWFzZQ1t6m5Atvu5r/SZdZx7q/rXjUKAF6/+W2hmMyHGdBzkY3m0JrSJvjJxULnZHNT0oGUmDvFnBYPQM1JY/LoBmN1IWhY7sj9a8lfwFWWzy/hBs7RgAAAABJRU5ErkJggg==')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/040ab5967b5b4f588dc8db80d21003fb/601b1/figurePricesData.webp 500w\",\"/static/040ab5967b5b4f588dc8db80d21003fb/80c60/figurePricesData.webp 1000w\",\"/static/040ab5967b5b4f588dc8db80d21003fb/e6a9b/figurePricesData.webp 1106w\"],\"sizes\":\"(max-width: 1106px) 100vw, 1106px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/040ab5967b5b4f588dc8db80d21003fb/0eb09/figurePricesData.png 500w\",\"/static/040ab5967b5b4f588dc8db80d21003fb/1263b/figurePricesData.png 1000w\",\"/static/040ab5967b5b4f588dc8db80d21003fb/22593/figurePricesData.png 1106w\"],\"sizes\":\"(max-width: 1106px) 100vw, 1106px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/040ab5967b5b4f588dc8db80d21003fb/22593/figurePricesData.png\",\"alt\":\"Figure prices dataset\",\"title\":\"Figure prices dataset\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`We then split our dataset into a train set and a test set and trained a linear regression (OLS regression) model\non the training data. Here’s how that looked like:`),mdx(ConditionalLazyPlot,_extends({},olsVsRidgeDataTrainTestOls,{mdxType:\"ConditionalLazyPlot\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"1103px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"63%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAABu0lEQVQ4y3VT2W6DMBBE4QwEfECTQMAcIQTaqqpSKY/9/7+ayktAlKYP1sJ6PMysB8MWJSyu4MgKUXpFmHZUd8cLwmMHf3+mfY17ttZ7xtS0hAI79WBZT0RRNpIuCQm3WlPfZAVVQz9oVTJ9hxWV2LCcNk32+9C/ClkB79BB1p/kjAi1mjh/w070COMBnjzDFoqWIyrY8i+Ro6ssYUUFgqJDe/9G1t5gbKJ8tJh2pM7lNcTxBpneiNjko2KLKdj8mUoFh9dg6UAjMoLDBSJ/JUJTW2QFtnmDSA1I8jtk9oUg6+BXZ2xPDWz2xLZQ86gMfQmxekd4vMDkBalw9zWclxHoiRaH7g6Rf8CLG+pZ4v+5Gl7SgOfDqJDp+JSjNV7S/CxR0Jdd3iCUA8L4FZ5oyOqSeLq8eYYsu9JBncc/FyCrWZknG+zkgOhBPCvmoyNjw4pH/q5zlpb5Wod3yqzDK+xkT2si09XQALJ96gms6/alocO6Wryg+bpxBVdWCPYt4djpCjep4Cct/OQ8zn/5p7hxTdXft6PFx++o+zqnwaOv380FTmdxwhPh2tZke3qm0Ks3Uj6RLXHrEf0A6Ys/2JEDqiMAAAAASUVORK5CYII=')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/026e7e9074d90660410322b8ef8dda0a/601b1/figurePricesDataOLS.webp 500w\",\"/static/026e7e9074d90660410322b8ef8dda0a/80c60/figurePricesDataOLS.webp 1000w\",\"/static/026e7e9074d90660410322b8ef8dda0a/8a7c0/figurePricesDataOLS.webp 1103w\"],\"sizes\":\"(max-width: 1103px) 100vw, 1103px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/026e7e9074d90660410322b8ef8dda0a/0eb09/figurePricesDataOLS.png 500w\",\"/static/026e7e9074d90660410322b8ef8dda0a/1263b/figurePricesDataOLS.png 1000w\",\"/static/026e7e9074d90660410322b8ef8dda0a/890d8/figurePricesDataOLS.png 1103w\"],\"sizes\":\"(max-width: 1103px) 100vw, 1103px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/026e7e9074d90660410322b8ef8dda0a/890d8/figurePricesDataOLS.png\",\"alt\":\"OLS for predicting figure prices\",\"title\":\"OLS for predicting figure prices\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`We then noticed that this model had a very low training error but a rather high testing error\nand thus we concluded that our linear regression model is overfit.\nWe then tried to come up with an imaginary, better model that was less overfit and looked more like this:`),mdx(ConditionalLazyPlot,_extends({},olsVsRidgeDataTrainTestOlsRidge,{mdxType:\"ConditionalLazyPlot\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"1106px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"63.6%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB/klEQVQ4y21T3ZqaMBBlEUQQIQl/KpCACijqdrsX/dqLff/HOv0mCGK3F0NCZuZk5syJYXMFsqWo4KUnuMkBXnKEE9Xw0iPc5NUohmItJjHmzs2gzyIsYXOJpLqDlz1E2SPYd2D5Bay4gBc94uoOIa9Iqnes4noCtNiQT0ZnBm38bYNN2sEMSphhMTnHwLmZD9+zKon1rsdm1+rKNSArzhC7G9b8Ak+0sOhmLrEUAxXfbNYigWcfv3C4/9GVG2ZQINx34PkJKy4RRA3S/SfC7AYzlHjTFSuYgdTtWY82rRGUSUTFFd3PL82/QcRrjvIGLC3AkhzZ8QjV/oBUH9iVPaJCIj4d4MYKrlBwI4mVUFiwocIgleg+vxCrGwwinTbr9IS3kKqoYKc1lomCF9cQ2yvk+2+IvIXYluBZAZHJh5XwYwkzVFoVmkOSAk11s21gsVLzRm3YbGjNZCVsVsNlLXzRY8UbXdmCS7iRgiOeXOopE4fBrtWANKDlf7SlEzj5KviiQxD18AQBy2961FMO87OWziiXucbG28d1QRXzCh5vEYgeDju8gE46pElrgvedHr8TVVhFtQ7yNR0SbnzQZ+T3tyc4vNb6fQEcf4hU2q+zBo6opudIUqAOCGgknlYaIsUTuM0GgU9Pb97Wv22L8opI3RDuzy++4bmOec/4v6RoQmYKoKCRAAAAAElFTkSuQmCC')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/bcd8acf2452585219b152cfe41dc8a87/601b1/figurePricesDataOLSRidge.webp 500w\",\"/static/bcd8acf2452585219b152cfe41dc8a87/80c60/figurePricesDataOLSRidge.webp 1000w\",\"/static/bcd8acf2452585219b152cfe41dc8a87/e6a9b/figurePricesDataOLSRidge.webp 1106w\"],\"sizes\":\"(max-width: 1106px) 100vw, 1106px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/bcd8acf2452585219b152cfe41dc8a87/0eb09/figurePricesDataOLSRidge.png 500w\",\"/static/bcd8acf2452585219b152cfe41dc8a87/1263b/figurePricesDataOLSRidge.png 1000w\",\"/static/bcd8acf2452585219b152cfe41dc8a87/22593/figurePricesDataOLSRidge.png 1106w\"],\"sizes\":\"(max-width: 1106px) 100vw, 1106px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/bcd8acf2452585219b152cfe41dc8a87/22593/figurePricesDataOLSRidge.png\",\"alt\":\"OLS vs Ridge for predicting figure prices\",\"title\":\"OLS vs Ridge for predicting figure prices\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`This imaginary model turned out to be ridge regression. We analyzed what exactly lead to our\nlinear regression model overfitting and we noticed that the main cause of overfitting were\n(absolutely) large model parameters.\nAfter discovering this insight, we developed a new loss function that penalizes large model parameters\nby adding a `,mdx(\"em\",{parentName:\"p\"},`penalty term`),` to our mean squared error.\nIt looked like this (where `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`m`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.43056em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`m`))))),` is the number of model parameters):`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`N`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`w`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nNew\\\\ Loss(y,y_{pred}) = MSE(y,y_{pred}) + \\\\sum_{i=1}^m{\\\\theta_i}\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`N`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02691em\"}},`w`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))))),mdx(\"p\",null,`There’s just one problem with this loss function. Since our model parameters can be negative, adding them might `,mdx(\"em\",{parentName:\"p\"},`decrease`),`\nour loss instead of increasing it.\nIn order to circumvent this, we can either square our model parameters or take their absolute values:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`N`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`w`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mn\",{parentName:\"mrow\"},`1`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"msubsup\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"msubsup\"},`θ`),mdx(\"mi\",{parentName:\"msubsup\"},`i`),mdx(\"mn\",{parentName:\"msubsup\"},`2`))),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"mi\",{parentName:\"mstyle\"},`R`),mdx(\"mi\",{parentName:\"mstyle\"},`i`),mdx(\"mi\",{parentName:\"mstyle\"},`d`),mdx(\"mi\",{parentName:\"mstyle\"},`g`),mdx(\"mi\",{parentName:\"mstyle\"},`e`),mdx(\"mi\",{parentName:\"mstyle\"},`M`),mdx(\"mi\",{parentName:\"mstyle\"},`S`),mdx(\"mi\",{parentName:\"mstyle\"},`E`)),mdx(\"mspace\",{parentName:\"mrow\",\"linebreak\":\"newline\"}),mdx(\"mi\",{parentName:\"mrow\"},`N`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`w`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mn\",{parentName:\"mrow\"},`2`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"mi\",{parentName:\"mstyle\"},`L`),mdx(\"mi\",{parentName:\"mstyle\"},`a`),mdx(\"mi\",{parentName:\"mstyle\"},`s`),mdx(\"mi\",{parentName:\"mstyle\"},`s`),mdx(\"mi\",{parentName:\"mstyle\"},`o`),mdx(\"mi\",{parentName:\"mstyle\"},`M`),mdx(\"mi\",{parentName:\"mstyle\"},`S`),mdx(\"mi\",{parentName:\"mstyle\"},`E`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nNew\\\\ Loss\\\\ 1(y,y_{pred}) = MSE(y,y_{pred}) + \\\\alpha \\\\sum_{i=1}^m{{\\\\color{#26a6ed}\\\\theta_i^2}} = {\\\\color{#26a6ed}RidgeMSE} \\\\\\\\\n\nNew\\\\ Loss\\\\ 2(y,y_{pred}) = MSE(y,y_{pred}) + \\\\alpha \\\\sum_{i=1}^m{{\\\\color{#26a6ed}|\\\\theta_i|}} = {\\\\color{#26a6ed}LassoMSE}\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`N`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02691em\"}},`w`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\",\"color\":\"#26a6ed\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4530000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#26a6ed\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#26a6ed\"}},`i`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#26a6ed\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"color\":\"#26a6ed\"}},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.247em\"}},mdx(\"span\",{parentName:\"span\"})))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8888799999999999em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.00773em\",\"color\":\"#26a6ed\"}},`R`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`d`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\",\"color\":\"#26a6ed\"}},`g`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\",\"color\":\"#26a6ed\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\",\"color\":\"#26a6ed\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\",\"color\":\"#26a6ed\"}},`E`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace newline\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`N`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02691em\"}},`w`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\",\"color\":\"#26a6ed\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#26a6ed\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#26a6ed\"}},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`∣`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.68333em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"color\":\"#26a6ed\"}},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\",\"color\":\"#26a6ed\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\",\"color\":\"#26a6ed\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\",\"color\":\"#26a6ed\"}},`E`))))))),mdx(\"p\",null,`The first function is the loss function of ridge regression, while the second one is the loss function of lasso regression.\nIn this article, we will focus our attention on the second loss function.`),mdx(\"p\",null,`If you are familiar with `,mdx(\"em\",{parentName:\"p\"},`norms`),` in math, then you could say that the lasso penalty is the `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`l`),mdx(\"mn\",{parentName:\"msub\"},`1`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`l_1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.01968em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),`-norm (or Manhattan norm)\nof our parameter vector `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\boldsymbol{\\\\theta}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.69444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))))))),`.\nThis is also the reason why this term is sometimes referred to as an `,mdx(\"em\",{parentName:\"p\"},`L1 regularization`),`.\nThis means that we can rewrite our new loss like this (a norm\nis denoted using two vertical lines, like this: ||):`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"msub\"},`1`)),mdx(\"mspace\",{parentName:\"mrow\",\"linebreak\":\"newline\"})),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nLassoMSE(y,y_{pred}) = MSE(y,y_{pred}) + \\\\alpha ||\\\\boldsymbol{\\\\theta}||_1 \\\\\\\\\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"0em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"}))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace newline\"}))))),mdx(\"p\",null,`In the article about ridge regression, we also went on and `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression/#ols-vs-ridge\"},`visualized`),` the difference\nbetween OLS regression and ridge regression. Since the visuals don’t change a lot for lasso,\nwe’ll skip this part for now.\nWe know the mathematical difference between ridge and lasso,\nbut what does this difference mean in practice?`),mdx(\"h2\",{\"id\":\"the-qualitative-difference-between-ridge-and-lasso\"},`The Qualitative Difference Between Ridge and Lasso`),mdx(\"p\",null,`When should you choose lasso over ridge (and vice versa)?\nAre there any differences in practice? Do lasso and ridge produce identical models?\nLet’s try to answer these questions by looking at a concrete example.\nWe will assume that we have some linear model with added regularization. Our linear model\nhas the parameter vector `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\boldsymbol{\\\\theta}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.69444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))))))),` with the following values:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`10`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`5`))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\boldsymbol{\\\\theta} = \\\\left[\\\\begin{array}{c}\n\n 10 \\\\\\\\\n 5\n\n\\\\end{array}\\\\right]`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.69444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`)))))))),mdx(\"p\",null,`Now let’s say that as part of some regularization procedure, we want to reduce the size of our model parameters.\nFor simplicity purposes, let’s assume that we’ll only change one of the parameters at a time.\nWhat happens when we use ridge and lasso regression, respectively? Let’s look at both cases.`),mdx(\"p\",null,mdx(\"strong\",{parentName:\"p\"},`Ridge`)),mdx(\"p\",null,`The ridge penalty is `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msubsup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msubsup\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"msubsup\"},`2`),mdx(\"mn\",{parentName:\"msubsup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"msup\"},`T`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`||\\\\boldsymbol{\\\\theta}||_2^2 = \\\\boldsymbol{\\\\theta}^T\\\\boldsymbol{\\\\theta}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.064108em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8141079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4518920000000004em\",\"marginLeft\":\"0em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.063em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.24810799999999997em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9256709999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9256709999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.1473400000000002em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.13889em\"}},`T`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))))))),`,\nso in our case this would be:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`R`),mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mi\",{parentName:\"mrow\"},`d`),mdx(\"mi\",{parentName:\"mrow\"},`g`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`10`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`5`))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"msup\"},`T`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"msubsup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msubsup\"},`θ`),mdx(\"mi\",{parentName:\"msubsup\"},`i`),mdx(\"mn\",{parentName:\"msubsup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mn\",{parentName:\"msup\"},`0`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mn\",{parentName:\"msup\"},`5`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`125`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nRidge\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n 10 \\\\\\\\\n 5\n\n\\\\end{array}\\\\right]) = \\\\boldsymbol{\\\\theta}^T\\\\boldsymbol{\\\\theta} = \\\\sum_{i=1}^m{\\\\theta_i^2} = 10^2 + 5^2 = 125\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.00773em\"}},`R`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`d`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9256709999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9256709999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.1473400000000002em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.13889em\"}},`T`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4530000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.247em\"}},mdx(\"span\",{parentName:\"span\"}))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9474379999999999em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8641079999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))))),mdx(\"p\",null,`If we reduce the first parameter by 1, our penalty will now look like this:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`R`),mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mi\",{parentName:\"mrow\"},`d`),mdx(\"mi\",{parentName:\"mrow\"},`g`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mstyle\",{parentName:\"mstyle\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`9`))))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`5`))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"msup\"},`T`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"msubsup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msubsup\"},`θ`),mdx(\"mi\",{parentName:\"msubsup\"},`i`),mdx(\"mn\",{parentName:\"msubsup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mstyle\",{parentName:\"msup\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`9`)),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mn\",{parentName:\"msup\"},`5`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`106`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nRidge\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n {\\\\color{#26a6ed}9} \\\\\\\\\n 5\n\n\\\\end{array}\\\\right]) = \\\\boldsymbol{\\\\theta}^T\\\\boldsymbol{\\\\theta} = \\\\sum_{i=1}^m{\\\\theta_i^2} = {\\\\color{#26a6ed}9}^2 + 5^2 = 106\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.00773em\"}},`R`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`d`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`9`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9256709999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9256709999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.1473400000000002em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.13889em\"}},`T`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4530000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.247em\"}},mdx(\"span\",{parentName:\"span\"}))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9474379999999999em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`9`)),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8641079999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`6`)))))),mdx(\"p\",null,`This means that by reducing our first model parameter from 10 to 9, we reduced our ridge penalty by 19,\nfrom 125 to 106. If we instead decreased the other parameter by 1, we would get this penalty:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`R`),mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mi\",{parentName:\"mrow\"},`d`),mdx(\"mi\",{parentName:\"mrow\"},`g`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`10`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mstyle\",{parentName:\"mstyle\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`4`)))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\"},mdx(\"msub\",{parentName:\"mi\"},mdx(\"mi\",{parentName:\"msub\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"msub\",\"mathvariant\":\"bold-italic\"},`i`))),mdx(\"mi\",{parentName:\"msup\"},`T`)),mdx(\"mi\",{parentName:\"mrow\"},`θ`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"msubsup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msubsup\"},`θ`),mdx(\"mi\",{parentName:\"msubsup\"},`i`),mdx(\"mn\",{parentName:\"msubsup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mn\",{parentName:\"msup\"},`0`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mstyle\",{parentName:\"msup\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`4`)),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`116`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nRidge\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n 10 \\\\\\\\\n {\\\\color{#26a6ed}4}\n\n\\\\end{array}\\\\right]) = \\\\boldsymbol{\\\\theta_i}^T\\\\theta = \\\\sum_{i=1}^m{\\\\theta_i^2} = 10^2 + {\\\\color{#26a6ed}4}^2 = 116\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.00773em\"}},`R`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`d`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`4`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.0756709999999998em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.33528199999999997em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03194em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9256709999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.1473400000000002em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.13889em\"}},`T`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4530000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.247em\"}},mdx(\"span\",{parentName:\"span\"}))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.9474379999999999em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8641079999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`4`)),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`6`)))))),mdx(\"p\",null,`This means that, by reducing our second model parameter from 5 to 4, we reduced our ridge penalty by 9,\nfrom 125 to 116. This is `,mdx(\"em\",{parentName:\"p\"},`less`),` than the reduction we got when we reduced our first model parameter!`),mdx(\"p\",null,`Since the ridge penalty squares the individual model parameters, large values are taken into account much more heavily than\nsmaller values. This means that our ridge regression model would `,mdx(\"em\",{parentName:\"p\"},`prioritize`),` minimizing large model parameters over small\nmodel parameters. This is usually a good thing because if our parameters are already small, they don’t need to be reduced even\nfurther. Let’s now take a look at how this situation looks like when using the lasso penalty.`),mdx(\"p\",null,mdx(\"strong\",{parentName:\"p\"},`Lasso`)),mdx(\"p\",null,`The lasso penalty is `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"msub\"},`1`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msubsup\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"msubsup\"},`∑`),mdx(\"mrow\",{parentName:\"msubsup\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"msubsup\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`||\\\\boldsymbol{\\\\theta}||_1 = \\\\sum_{i=1}^m{|\\\\theta_i|}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"0em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.104002em\",\"verticalAlign\":\"-0.29971000000000003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol small-op\",\"style\":{\"position\":\"relative\",\"top\":\"-0.0000050000000000050004em\"}},`∑`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.804292em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.40029em\",\"marginLeft\":\"0em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.2029em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.29971000000000003em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`)))))),`,\nso in our case this would be:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`10`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`5`))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"mrow\"},`10`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"mrow\"},`5`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`15`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nLasso\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n 10 \\\\\\\\\n 5\n\n\\\\end{array}\\\\right]) = \\\\sum_{i=1}^m{|\\\\theta_i|} = |10| + |5| = 15\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))))),mdx(\"p\",null,`If we reduce the first parameter by 1, our penalty will now look like this:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mstyle\",{parentName:\"mstyle\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`9`))))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`5`))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`9`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"mrow\"},`5`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`14`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nLasso\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n {\\\\color{#26a6ed}9} \\\\\\\\\n 5\n\n\\\\end{array}\\\\right]) = \\\\sum_{i=1}^m{|\\\\theta_i|} = |{\\\\color{#26a6ed}9}| + |5| = 14\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`9`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`9`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`5`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`4`)))))),mdx(\"p\",null,`This means that by reducing our first model parameter from 10 to 9, we’ve reduced our ridge penalty by 1,\nfrom 15 to 14. If we instead decreased the other parameter, we would get this penalty:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mtext\",{parentName:\"mrow\"},` `),mdx(\"mi\",{parentName:\"mrow\"},`P`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`n`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`l`),mdx(\"mi\",{parentName:\"mrow\"},`t`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`10`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mstyle\",{parentName:\"mstyle\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`4`)))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mn\",{parentName:\"mrow\"},`10`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#26a6ed\"},mdx(\"mn\",{parentName:\"mstyle\"},`4`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`14`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nLasso\\\\ Penalty(\\\\left[\\\\begin{array}{c}\n\n 10 \\\\\\\\\n {\\\\color{#26a6ed}4}\n\n\\\\end{array}\\\\right]) = \\\\sum_{i=1}^m{|\\\\theta_i|} = |10| + |{\\\\color{#26a6ed}4}| = 14\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.13889em\"}},`P`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`n`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.01968em\"}},`l`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`t`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`4`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#26a6ed\"}},`4`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`1`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`4`)))))),mdx(\"p\",null,`This means that by reducing our first model parameter from 10 to 9, we reduced our ridge penalty by 1,\nfrom 15 to 14. This is `,mdx(\"em\",{parentName:\"p\"},`equal`),` to the first reduction!`),mdx(\"p\",null,`Since the lasso penalty consists of the absolute model parameters, large values are not taken into account more strongly\nthan smaller values. This means that our lasso penalty would not prioritize minimizing any particular\nmodel parameter, unlike the ridge penalty, which prioritizes large parameters.`),mdx(\"h3\",{\"id\":\"parameter-sparsity-of-lasso\"},`Parameter Sparsity of Lasso`),mdx(\"p\",null,`One consequence of this is that with ridge regression, weights can get very very small, but they will never be zero.\nThis is because if we square a number between 1 and 0, the square will be smaller than our original number.\nIn other words, the ridge penalty gets smaller and smaller the closer we get to zero.\nAnd this shrinking of the ridge penalty is amplified when we get closer to zero.\nWith lasso however, this loss shrinking is constant.\nThis is why, when we use lasso regression, some weights (more precisely, the least useful ones,\nbecause they also contribute least to minimizing the loss) might be set all the way to zero.`),mdx(InfoTip,{mdxType:\"InfoTip\"},mdx(\"p\",null,`In technical literature, it is sometimes written that lasso leads to `,mdx(\"em\",{parentName:\"p\"},`sparse weights`),` or\n`,mdx(\"em\",{parentName:\"p\"},`parameter sparsity`),`. A sparse matrix is a matrix with many zeros, so in this context,\nthe term `,mdx(\"em\",{parentName:\"p\"},`sparse weights`),` refers to the property of lasso that some weights will be set all the way to zero.`)),mdx(\"p\",null,`Let’s look at a small example.\nWe want to predict the price of a figure based on the age of the figure, as before.\nBut this time, we also take into account the initial price of the figure (`,mdx(\"em\",{parentName:\"p\"},`init price`),`), the number\nof figures produced in the initial year of release (`,mdx(\"em\",{parentName:\"p\"},`#produced`),`), the average temperature in the country the\nfigure was manufactured in (`,mdx(\"em\",{parentName:\"p\"},`temp`),`), as well as the average number of pets per household in Europe (`,mdx(\"em\",{parentName:\"p\"},`PPH`),`).\nIf we write this down as a function, we would have:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mi\",{parentName:\"mrow\"},`c`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mtext\",{parentName:\"mrow\",\"mathvariant\":\"bold\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`1`)),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mtext\",{parentName:\"mrow\"},`age`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`2`)),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mtext\",{parentName:\"mrow\"},`init. price`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`3`)),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mtext\",{parentName:\"mrow\"},`#produced`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`4`)),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mtext\",{parentName:\"mrow\"},`temp`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`5`)),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mtext\",{parentName:\"mrow\"},`PPH`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`0`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nprice(\\\\textbf{x}) = \\\\theta_1 \\\\cdot \\\\text{age} + \\\\theta_2 \\\\cdot \\\\text{init. price} + \\\\theta_3 \\\\cdot \\\\text{\\\\#produced} + \\\\theta_4 \\\\cdot \\\\text{temp} + \\\\theta_5 \\\\cdot \\\\text{PPH} + \\\\theta_0\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`c`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord textbf\"},`x`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.7777700000000001em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`age`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8623000000000001em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`init. price`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`3`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8888799999999999em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`#produced`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`4`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.80952em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`temp`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.76666em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`PPH`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`0`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"}))))))))))),mdx(\"p\",null,`The age, initial price, and the number of figures produced all seem to be quite relevant features.\nIn contrast, the average temperature and the number of pets per household seem completely irrelevant to this regression problem.\nHow would knowing the average amount of pets per household in Europe help us predict figure prices?\nThese last features are pretty useless in this regard.\nIf we use ridge regression for this problem,\nnone of the weights `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`1`)),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`2`)),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`3`)),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`4`)),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_1, \\\\theta_2, \\\\theta_3, \\\\theta_4,`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8888799999999999em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`3`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`4`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`))))),` and `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`5`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` would be reduced to 0.\nThey would probably be reduced to very small values, but they would not be set to 0, even though they are completely useless.\nHowever, when we use lasso regression, the weights `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`4`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_4`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`4`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` and `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`5`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`5`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` that control the influence\nof the average temperature and the number of pets per household in Europe, would be set to exactly 0.\nThis is nice because it allows us to completely eliminate some features from our dataset and focus only on the important ones.\nIn most scenarios, we don’t actually know which features are useful and which are not.\nWith lasso, when a weight is equal to exactly 0, we can remove it from our dataset and our model\nwill still perform exactly the same as before.`),mdx(\"p\",null,`From this we can conclude that lasso regression is `,mdx(\"em\",{parentName:\"p\"},`particularly`),` well suited when we want to minimize the number\nof features our model should use. If we think that a lot of features in our model might be rather unhelpful for our\nmodel, then using lasso is a great choice. Conversely, if we think that most features in our dataset could be really\nhelpful, ridge might have a slight advantage over lasso. In most cases, however, these two algorithms will produce\nvery similar results. So which one should you choose if you don’t know anything about the usefulness of your features?\nThen your best bet is to just use `,mdx(\"em\",{parentName:\"p\"},`both`),` lasso and ridge and determine the best value for `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`α`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\alpha`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.43056em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`))))),`\nusing cross-validation. There is also a model called `,mdx(\"em\",{parentName:\"p\"},`elastic net`),` which, in a nutshell, is just OLS\nregression with both a ridge penalty and a lasso penalty attached to it. If you’re interested in elastic net,\nthe third article of this series, `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/elastic_net_regression\"},`Elastic Net Regression Explained, Step by Step`),`,\nhas all of the relevant details you need to know! But back to lasso. How can we actually determine\nwhat model parameters `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\boldsymbol{\\\\theta}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.69444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))))))),` we should use?`),mdx(\"h2\",{\"id\":\"solving-lasso-regression\"},`Solving Lasso Regression`),mdx(\"p\",null,`In `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression/#solving-ridge-regression\"},`this section`),` of the article about ridge regression, we’ve looked at the ways to solve ridge regression.\nIt turned out that, to solve ridge, we can just use the same techniques that we used to solve\nregular OLS regression. Lasso can’t be that much different, right? Well, it is, actually.\nThe bad news is that there is no closed-form solution for lasso, meaning there is no normal\nequation we can solve to immediately get all of our ideal model parameters. But why?\nWhat is it about lasso that makes things more complicated?`),mdx(\"p\",null,`Recall that we can derive a normal equation by finding the minimum of our loss\nfunction, i.e. differentiating it, setting it equal to zero, and then solving for\nour model parameters. However, our lasso penalty makes things a little bit more\ndifficult. To understand why, let’s look at a small example.\nLet’s consider the simple function `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`f`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\"},`x`),mdx(\"mn\",{parentName:\"msup\"},`2`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`f(x) = x^2`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8141079999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8141079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.063em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`)))))))))))),`.\nTo find the minimum of `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`f`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`f`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8888799999999999em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`))))),`, we take the first derivative of it, we set it equal to zero, and then we solve for `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`x`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`x`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.43056em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`))))),`.\nWith this we get that our minimum is at x=0, easy.\nNow let’s consider the following function, which is the absolute function: `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`g`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`g(x) = |x|`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`))))),`.\nThe function looks like this:`),mdx(ConditionalLazyPlot,_extends({},absoluteFunctionBase,{mdxType:\"ConditionalLazyPlot\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"967px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"62.2%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAABYlAAAWJQFJUiTwAAABQUlEQVQoz4WT626DMAyFkVYSoNtaLgFaiVugA6Rpe/+nO5MN4dZU+2HFJNaX4+PguGGF07UEran+Rdr8IKm+OSgX0Xpuas035/O+CYc2RVRDHA5scQRzPl9ggoFeohHcR7hRwwU2GCl92mcxDVz1gJf1+LwPE1DGDYK0g8hGiFhbodTFEUYCZDbshDhvl4KBfqLhhjVkPsK1QHcKNzCxgXHLdDO1TEDjy6S0xelaPAOpJtYQWc8C3HD1khUaoMcKq6mAbMhHiGSFUh3nC6yyWsMte3HDsT8sp5aSbjKfLorbFRbah7cMRR5NN1MkqOogVQeR9i9fwQ7oqxbvWcdtybheVjm/Ty8f4N9GzuW2ZskPQDOhj9sXfKXZqyBtcc46nC4FAqVxTluuIWvocqrxEsofuwfvbH+nV8F/Eg+l/Df+APqqI/kw5lGFAAAAAElFTkSuQmCC')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/380ba17ae7aaf4cd954668606ca548b0/601b1/absoluteFunctionPlot.webp 500w\",\"/static/380ba17ae7aaf4cd954668606ca548b0/0a18a/absoluteFunctionPlot.webp 967w\"],\"sizes\":\"(max-width: 967px) 100vw, 967px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/380ba17ae7aaf4cd954668606ca548b0/0eb09/absoluteFunctionPlot.png 500w\",\"/static/380ba17ae7aaf4cd954668606ca548b0/3ccd1/absoluteFunctionPlot.png 967w\"],\"sizes\":\"(max-width: 967px) 100vw, 967px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/380ba17ae7aaf4cd954668606ca548b0/3ccd1/absoluteFunctionPlot.png\",\"alt\":\"Absolute Function\",\"title\":\"Absolute Function\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`We can’t use the same approach for this function, because `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`g`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`g`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.625em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`))))),` is not differentiable at x=0.\nThe slope of `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`g`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`g`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.625em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`))))),` is 1 for all values greater than 0, and -1 for all values smaller than 0.\nBut what is the slope of `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`g`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`g`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.625em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`))))),` at x=0? It is undefined. So we can’t actually differentiate our function\n`,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`g`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`g`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.625em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`g`))))),` at x=0.`),mdx(\"p\",null,`Now back to lasso. We want to derive a closed-form solution for the lasso-MSE:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nLassoMSE(y,y_{pred}) = MSE(y,y_{pred}) + \\\\sum_{i=1}^m{|\\\\theta_i|}\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.036108em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.929066em\",\"verticalAlign\":\"-1.277669em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`))))))),mdx(\"p\",null,`But this function also contains absolute values!\nRecall the sum rule for derivatives:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`h`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`2`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mn\",{parentName:\"mrow\"},`3`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\"},`x`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mspace\",{parentName:\"mrow\",\"linebreak\":\"newline\"}),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\"},`h`),mdx(\"mo\",{parentName:\"msup\",\"mathvariant\":\"normal\",\"lspace\":\"0em\",\"rspace\":\"0em\"},`′`)),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mtext\",{parentName:\"mrow\"},`derivative of `),mdx(\"mn\",{parentName:\"mrow\"},`2`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mtext\",{parentName:\"mrow\"},`derivative of `),mdx(\"mn\",{parentName:\"mrow\"},`3`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`2`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mn\",{parentName:\"mrow\"},`6`),mdx(\"mi\",{parentName:\"mrow\"},`x`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`h(x) = 2x + 3x^2 \\\\\\\\\nh'(x) = (\\\\text{derivative of}\\\\ 2x) + (\\\\text{derivative of}\\\\ 3x) = 2 + 6x`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`h`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.72777em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.8641079999999999em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`3`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`))))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace newline\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.051892em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`h`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.801892em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`′`))))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`derivative of`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord text\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`derivative of`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\"},` `),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`3`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.72777em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`6`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`)))))),mdx(\"p\",null,`In the case of our lasso loss, we have a multivariate function, since our inputs `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`y`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`y`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.625em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`))))),` and `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`)))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`y_{pred}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.716668em\",\"verticalAlign\":\"-0.286108em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` are vectors (and because there’s two of them),\nbut we can just use the `,mdx(\"em\",{parentName:\"p\"},`multivariate`),` sum rule for derivatives:`),mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mtable\",{parentName:\"semantics\",\"rowspacing\":\"0.24999999999999992em\",\"columnalign\":\"right left\",\"columnspacing\":\"0em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"bold-italic\"},`θ`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`)))),mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mrow\",{parentName:\"mrow\"}),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)))))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`L`),mdx(\"mi\",{parentName:\"mrow\"},`a`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`s`),mdx(\"mi\",{parentName:\"mrow\"},`o`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`)))),mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mrow\",{parentName:\"mrow\"}),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#FF8900\"},mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mstyle\",\"stretchy\":\"false\"},`(`),mdx(\"munderover\",{parentName:\"mstyle\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∣`)),mdx(\"mo\",{parentName:\"mstyle\",\"stretchy\":\"false\"},`)`)))))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"}))),mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"true\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mrow\",{parentName:\"mrow\"}),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`M`),mdx(\"mi\",{parentName:\"mrow\"},`S`),mdx(\"mi\",{parentName:\"mrow\"},`E`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`y`),mdx(\"mrow\",{parentName:\"msub\"},mdx(\"mi\",{parentName:\"mrow\"},`p`),mdx(\"mi\",{parentName:\"mrow\"},`r`),mdx(\"mi\",{parentName:\"mrow\"},`e`),mdx(\"mi\",{parentName:\"mrow\"},`d`))),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mi\",{parentName:\"mrow\"},`α`),mdx(\"mo\",{parentName:\"mrow\"},`⋅`),mdx(\"munderover\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"munderover\"},`∑`),mdx(\"mrow\",{parentName:\"munderover\"},mdx(\"mi\",{parentName:\"mrow\"},`i`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`1`)),mdx(\"mi\",{parentName:\"munderover\"},`m`)),mdx(\"mstyle\",{parentName:\"mrow\",\"mathcolor\":\"#FF8900\"},mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mstyle\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∣`),mdx(\"msub\",{parentName:\"mstyle\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mi\",{parentName:\"mstyle\",\"mathvariant\":\"normal\"},`∣`),mdx(\"mo\",{parentName:\"mstyle\",\"stretchy\":\"false\"},`)`))))))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\begin{aligned}\nLassoMSE(y,y_{pred},\\\\boldsymbol{\\\\theta}) &= MSE(y,y_{pred}) + \\\\alpha \\\\cdot \\\\sum_{i=1}^m{|\\\\theta_i|} \\\\\\\\\n\n\\\\nabla(LassoMSE) &= \\\\nabla(MSE(y,y_{pred})) + \\\\alpha \\\\cdot {\\\\color{#FF8900}\\\\nabla(\\\\sum_{i=1}^m{|\\\\theta_i|})} \\\\\\\\\n\n&= \\\\nabla(MSE(y,y_{pred})) + \\\\alpha \\\\cdot \\\\sum_{i=1}^m{{\\\\color{#FF8900}\\\\nabla(|\\\\theta_i|)}}\n\\\\end{aligned}`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"9.687198000000002em\",\"verticalAlign\":\"-4.593599000000001em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"5.093599000000001em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-7.093599000000001em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord boldsymbol\",\"style\":{\"marginRight\":\"0.03194em\"}},`θ`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.8645330000000007em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`L`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`a`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`s`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`o`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-0.6354669999999991em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"}))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"4.593599000000001em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-l\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"5.093599000000001em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-7.093599000000001em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∣`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.8645330000000007em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\",\"style\":{\"color\":\"#FF8900\"}},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#FF8900\"}},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\",\"style\":{\"color\":\"#FF8900\"}},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"color\":\"#FF8900\"}},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\",\"style\":{\"color\":\"#FF8900\"}},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#FF8900\"}},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\",\"color\":\"#FF8900\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#FF8900\"}},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∣`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\",\"style\":{\"color\":\"#FF8900\"}},`)`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-0.6354669999999991em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.6513970000000002em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10903em\"}},`M`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`S`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.05764em\"}},`E`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.3361079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.03588em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`p`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.02778em\"}},`r`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`e`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`d`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.286108em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`⋅`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-limits\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.6513970000000002em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-1.872331em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel mtight\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.050005em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mop op-symbol large-op\"},`∑`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-4.3000050000000005em\",\"marginLeft\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3.05em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`m`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.277669em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\",\"style\":{\"color\":\"#FF8900\"}},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\",\"color\":\"#FF8900\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\",\"style\":{\"color\":\"#FF8900\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"color\":\"#FF8900\"}},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\",\"style\":{\"color\":\"#FF8900\"}},`∣`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\",\"style\":{\"color\":\"#FF8900\"}},`)`)))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"4.593599000000001em\"}},mdx(\"span\",{parentName:\"span\"})))))))))))),mdx(\"p\",null,`Here, `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\nabla`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.68333em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`))))),` is the `,mdx(\"em\",{parentName:\"p\"},`gradient`),`-operator. The word “gradient” sounds really fancy but all that `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`f`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\nabla(f)`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`))))),` does is provide\nus with all of the partial derivatives of `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`x`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`x`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.43056em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`))))),` in a vector.\nHere’s a little example:`),mdx(Collapsible,{title:\"Example Gradient Calculation\",mdxType:\"Collapsible\"},mdx(\"div\",{\"className\":\"math math-display\"},mdx(\"span\",{parentName:\"div\",\"className\":\"katex-display\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\",\"display\":\"block\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`f`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\",\"separator\":\"true\"},`,`),mdx(\"mi\",{parentName:\"mrow\"},`y`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`2`),mdx(\"mi\",{parentName:\"mrow\"},`x`),mdx(\"mo\",{parentName:\"mrow\"},`+`),mdx(\"mn\",{parentName:\"mrow\"},`3`),mdx(\"msup\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msup\"},`y`),mdx(\"mn\",{parentName:\"msup\"},`2`)),mdx(\"mspace\",{parentName:\"mrow\",\"linebreak\":\"newline\"}),mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∇`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`(`),mdx(\"mi\",{parentName:\"mrow\"},`f`),mdx(\"mo\",{parentName:\"mrow\",\"stretchy\":\"false\"},`)`),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mfrac\",{parentName:\"mstyle\"},mdx(\"mrow\",{parentName:\"mfrac\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∂`),mdx(\"mi\",{parentName:\"mrow\"},`f`)),mdx(\"mrow\",{parentName:\"mfrac\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∂`),mdx(\"mi\",{parentName:\"mrow\"},`x`)))))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mfrac\",{parentName:\"mstyle\"},mdx(\"mrow\",{parentName:\"mfrac\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∂`),mdx(\"mi\",{parentName:\"mrow\"},`f`)),mdx(\"mrow\",{parentName:\"mfrac\"},mdx(\"mi\",{parentName:\"mrow\",\"mathvariant\":\"normal\"},`∂`),mdx(\"mi\",{parentName:\"mrow\"},`y`))))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mrow\",{parentName:\"mrow\"},mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`[`),mdx(\"mtable\",{parentName:\"mrow\",\"rowspacing\":\"0.15999999999999992em\",\"columnalign\":\"center\",\"columnspacing\":\"1em\"},mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mn\",{parentName:\"mstyle\"},`2`)))),mdx(\"mtr\",{parentName:\"mtable\"},mdx(\"mtd\",{parentName:\"mtr\"},mdx(\"mstyle\",{parentName:\"mtd\",\"scriptlevel\":\"0\",\"displaystyle\":\"false\"},mdx(\"mrow\",{parentName:\"mstyle\"},mdx(\"mn\",{parentName:\"mrow\"},`6`),mdx(\"mi\",{parentName:\"mrow\"},`y`)))))),mdx(\"mo\",{parentName:\"mrow\",\"fence\":\"true\"},`]`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\nf(x,y) = 2x + 3y^2 \\\\\\\\\n\n\\\\nabla(f) = \\\\left[\\\\begin{array}{c}\n\n \\\\frac{\\\\partial f}{\\\\partial x} \\\\\\\\\n \\\\frac{\\\\partial f}{\\\\partial y}\n\n\\\\end{array}\\\\right] =\n\n\\\\left[\\\\begin{array}{c}\n\n 2 \\\\\\\\\n 6y\n\n\\\\end{array}\\\\right]\n\n`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mpunct\"},`,`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.16666666666666666em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.72777em\",\"verticalAlign\":\"-0.08333em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\"},`x`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mbin\"},`+`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2222222222222222em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1.0585479999999998em\",\"verticalAlign\":\"-0.19444em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`3`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.8641079999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.113em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`2`))))))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace newline\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"1em\",\"verticalAlign\":\"-0.25em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`∇`),mdx(\"span\",{parentName:\"span\",\"className\":\"mopen\"},`(`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose\"},`)`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"3.0000299999999998em\",\"verticalAlign\":\"-1.25003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size4\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.60277em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.670554em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen nulldelimiter\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"mfrac\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9322159999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.6550000000000002em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"marginRight\":\"0.05556em\"}},`∂`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`x`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.23em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"frac-line\",\"style\":{\"borderBottomWidth\":\"0.04em\"}})),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.446108em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"marginRight\":\"0.05556em\"}},`∂`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.345em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose nulldelimiter\"})))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.378338em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen nulldelimiter\"}),mdx(\"span\",{parentName:\"span\",\"className\":\"mfrac\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9322159999999999em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.6550000000000002em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"marginRight\":\"0.05556em\"}},`∂`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`)))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.23em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"frac-line\",\"style\":{\"borderBottomWidth\":\"0.04em\"}})),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.446108em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\",\"style\":{\"marginRight\":\"0.05556em\"}},`∂`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\",\"style\":{\"marginRight\":\"0.10764em\"}},`f`))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.481108em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose nulldelimiter\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.10277em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size4\"},`]`))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"2.40003em\",\"verticalAlign\":\"-0.95003em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"minner\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mopen delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`[`)),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mtable\"},mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"col-align-c\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"1.45em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-3.61em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`2`))),mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.4099999999999997em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"3em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`6`),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.03588em\"}},`y`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.9500000000000004em\"}},mdx(\"span\",{parentName:\"span\"}))))),mdx(\"span\",{parentName:\"span\",\"className\":\"arraycolsep\",\"style\":{\"width\":\"0.5em\"}}))),mdx(\"span\",{parentName:\"span\",\"className\":\"mclose delimcenter\",\"style\":{\"top\":\"0em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"delimsizing size3\"},`]`))))))))),mdx(\"p\",null,`Gradients are explained\nin more detail in this `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_math/gradient_descent_for_linear_regression/#the-gradient\"},`section of the article about gradient descent for linear regresion`),`,\nso if you’d like to read more about this, check out that article.`),mdx(\"p\",null,`So what we’re doing here is taking the derivative of our MSE, adding that to the derivative\nof our lasso penalty, summing those two up, and what we get is the derivative of our lasso-MSE.\nThe orange part in the gradient above\nis the derivative of our lasso penalty, which is the derivative\nof a sum of absolute values. As we just saw, we can’t differentiate these absolute terms\nfor `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mi\",{parentName:\"msub\"},`i`)),mdx(\"mo\",{parentName:\"mrow\"},`=`),mdx(\"mn\",{parentName:\"mrow\"},`0`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_i=0`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.31166399999999994em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal mtight\"},`i`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mrel\"},`=`),mdx(\"span\",{parentName:\"span\",\"className\":\"mspace\",\"style\":{\"marginRight\":\"0.2777777777777778em\"}})),mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.64444em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},`0`))))),` , so we can’t compute the gradient here!`),mdx(\"p\",null,`This is the reason why there is no general closed-form solution for lasso. This is by no means a full, mathematically\nrigorous proof, but it should be more than enough to understand the underlying problem.\nSo what can we do? We can’t just `,mdx(\"em\",{parentName:\"p\"},`ignore`),` the point at x=0, since it’s a pretty important one.\nWe can’t compute a closed-form solution, so we’ll have to use a different method to solve lasso.\nIn particular, we are going to use gradient descent to do so! Or rather, we are going to use some slight\nvariations of gradient descent, because regular gradient descent also needs the gradient of our function (as the name would suggest),\nwhich, as we just saw, can’t be computed.\nThe two most popular variations of gradient descent that are used to solve lasso are\n`,mdx(\"em\",{parentName:\"p\"},`coordinate descent`),` and `,mdx(\"em\",{parentName:\"p\"},`subgradient descent`),`.\nIn a nutshell,\ncoordinate descent updates one parameter (or `,mdx(\"em\",{parentName:\"p\"},`coordinate`),`) at a time,\nwhile subgradient descent uses a `,mdx(\"em\",{parentName:\"p\"},`subgradient`),` instead of a regular gradient, which, unlike the regular gradient,\nwe can compute for the lasso-MSE.`),mdx(\"p\",null,`When I first wrote this article, I explained both subgradient and coordinate descent,\nshowed how to implement them from scratch and then I also visualized them.\nThis however lead to the article taking over 40 minutes to read in it’s entirety.\nSo instead of doing that, let’s get straight to the visuals and compare subgradient and coordinate descent\non a graphical level! (Don’t worry, we’ll get back to the other parts in a moment)`),mdx(\"h3\",{\"id\":\"visualizing-subgradient-descent-and-coordinate-descent\"},`Visualizing Subgradient Descent and Coordinate Descent`),mdx(\"p\",null,`Before looking at the visualizations, let’s first ask ourselves what we should be expecting.\nFor subgradient descent, it should probably not look too different from regular gradient descent,\nbecause our subgradient will be `,mdx(\"em\",{parentName:\"p\"},`similar`),` to a supposed regular gradient.\nBelow you can see a 3-dimensional plot where for every combination of\ninput parameters `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`1`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`1`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` and `,mdx(\"span\",{parentName:\"p\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"msub\",{parentName:\"mrow\"},mdx(\"mi\",{parentName:\"msub\"},`θ`),mdx(\"mn\",{parentName:\"msub\"},`0`))),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\theta_0`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.84444em\",\"verticalAlign\":\"-0.15em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.02778em\"}},`θ`),mdx(\"span\",{parentName:\"span\",\"className\":\"msupsub\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-t vlist-t2\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.30110799999999993em\"}},mdx(\"span\",{parentName:\"span\",\"style\":{\"top\":\"-2.5500000000000003em\",\"marginLeft\":\"-0.02778em\",\"marginRight\":\"0.05em\"}},mdx(\"span\",{parentName:\"span\",\"className\":\"pstrut\",\"style\":{\"height\":\"2.7em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"sizing reset-size6 size3 mtight\"},mdx(\"span\",{parentName:\"span\",\"className\":\"mord mtight\"},`0`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-s\"},`​`)),mdx(\"span\",{parentName:\"span\",\"className\":\"vlist-r\"},mdx(\"span\",{parentName:\"span\",\"className\":\"vlist\",\"style\":{\"height\":\"0.15em\"}},mdx(\"span\",{parentName:\"span\"})))))))))),` (I’ve named them m and b in the plot, to make them more easily distinguishable),\nthe lasso-MSE is plotted. By pressing the buttons at the top, you can perform one iteration step of subgradient descent.\nAfter the first ten steps, each button press cycles through more than just one step at a time, so that you don’t have\nto press one button 300 times.\nLet’s take a look:`),mdx(ConditionalLazyPlotInteractive,_extends({},lassoSubGDHistPlot,{mdxType:\"ConditionalLazyPlotInteractive\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"1015px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"80.4%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAQCAYAAAAWGF8bAAAACXBIWXMAABYlAAAWJQFJUiTwAAADa0lEQVQ4y3WU228UZRiHpzt7mJndzh5mD7OzMzs7M1u2u+2WYulhCQtVyxZI2uKFBgTDBdHYGGusMUqNwTNYSaCSQJpAsYYSjEbBwA0Y44UxXnDj3+If8JiZbaX1cPFmvvnyfs97+L73J0TSNuFUme3fLfP/o9vWkf9Y96gmrWcOkSnXCfWaCP8HCmAZm55eEzFp7djf8otmKgixAt/+uMbU7HMIcnEn0HfYco5pDmKyjFnbg1ZuIP4j227ACoJU4Pqda+zrHN0EpjZT99OVdKSsi5x3Cads5KzDl+sXaB85jKCUkLJOAAksSMAOzqzc+oKJzmEE2UCIBVF0hloHOXh0prspFYnnHE6fnWftt5uceHcesddCKVSRs24Q1K8gEgCLnLvxESNTHXqUUrfkmOZSGRjllz/ucfXuZU6de5P51fMsfH+D52+uMrt6lWxjlIhqIuerKDkPOed2gXKRV658QmPyECHFQBBVk0y5gSAZnFw8w8M/f+e9x4944cEDDlzfYPDyLeor32CdOUs0WULJV5Fz1W6/A6DOzPJ5vHYHMW4gJAp9ZKwB1GINrdLgtft3mP3pVwbXf6bv4j0qS7cxFr6mtHibdPsloqqOnDaJad0LDCk6Yx8uY7ePEE2UEPxeuM0WOXc3IalA/eTLtO4/pvz5I+w37tJ/6iu8Y2u4s2u4JzYoPvs2amOKSNbrAuM6tXcuoo9PE1YKfskWBXc3OWeIeFCOi/PBOpWlh3jH1umfXKU+cQln7wrViSvU5r7DnFkmspmhGNcpzS+THekQjhe6lxJSTZS8h1LoQ0qWSI9MU3n9B2pTa1SGL7BrZJHqngWMwfepjl5CKY4R6tWJZBzEhE7x+Mdkh6YIJ/QucMus/r1Y9TFkrYw2fpza7AbW8DJPjZ9m//4XyTWXSFrTFNwBNLtJKGkRVouUOktk+58m3LsN6I+XpDnYjXEy5UFEtUimOYfT/gx77FOKzUUSpUkyVj/OYAs55wWPX1QNrLG30NwDAXzH6PnQeKEv6GnKbBDLWMSNOil7H0p+IIDp3jD+y/DbFElXEJMGRv1Vck6bcNL4tzhIwSQ4mLUR1FIdKesRTZtodiOAbc35E4UySdlzSPkmkdR2tdkmYb4o+A83ofdtBvFI6Lv+FoUnsM2zGY+wP9tpm78Akljb/F8luIQAAAAASUVORK5CYII=')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/bc77d74c927719b57bcfe679bb2e186a/601b1/lassoSubGDHist.webp 500w\",\"/static/bc77d74c927719b57bcfe679bb2e186a/80c60/lassoSubGDHist.webp 1000w\",\"/static/bc77d74c927719b57bcfe679bb2e186a/5599d/lassoSubGDHist.webp 1015w\"],\"sizes\":\"(max-width: 1015px) 100vw, 1015px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/bc77d74c927719b57bcfe679bb2e186a/0eb09/lassoSubGDHist.png 500w\",\"/static/bc77d74c927719b57bcfe679bb2e186a/1263b/lassoSubGDHist.png 1000w\",\"/static/bc77d74c927719b57bcfe679bb2e186a/4d2c8/lassoSubGDHist.png 1015w\"],\"sizes\":\"(max-width: 1015px) 100vw, 1015px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/bc77d74c927719b57bcfe679bb2e186a/4d2c8/lassoSubGDHist.png\",\"alt\":\"Subgradient Descent for lasso\",\"title\":\"Subgradient Descent for lasso\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`Alright, just as expected!\nSo subgradient descent proves to be a working alternative when regular gradient descent doesn’t work. Nice!\nBut using subgradient descent in the context of lasso regression has one `,mdx(\"strong\",{parentName:\"p\"},`fatal`),` flaw, which is also the reason why scikit-learn only provides\nan implementation of coordinate descent, but not one of subgradient descent for its `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),`-class.\nThe worst part is that in many books (even in really good ones!) this part is often overlooked when talking\nabout subgradient descent for lasso. We’ll have to take a deeper look into subgradient descent to really understand\nwhat’s going on. So if you want to understand how subgradient descent works\nand why you `,mdx(\"em\",{parentName:\"p\"},`shouldn’t`),` use it to solve lasso regression, read the article `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_math/subgradient_descent\"},`Subgradient Descent Explained, Step by Step`),`\nto find out! As far as I know, this is currently the only article on the web that covers this topic in\nsuch great detail, so it will definitely be worth your time!`),mdx(\"p\",null,`Ok, but what about coordinate descent?\nSince we’re only looking at one coordinate at a time, the descent should look a lot more `,mdx(\"em\",{parentName:\"p\"},`zigzaggy`),` than\nregular gradient descent. Let’s again make a plot where you can click buttons to control\nthe current iteration of gradient descent. Have a look:`),mdx(ConditionalLazyPlotInteractive,_extends({},lassoCDHistPlot,{mdxType:\"ConditionalLazyPlotInteractive\"}),mdx(\"p\",null,mdx(\"span\",{parentName:\"p\",\"className\":\"gatsby-resp-image-wrapper\",\"style\":{\"position\":\"relative\",\"display\":\"block\",\"marginLeft\":\"auto\",\"marginRight\":\"auto\",\"maxWidth\":\"984px\"}},`\n `,mdx(\"span\",{parentName:\"span\",\"className\":\"gatsby-resp-image-background-image\",\"style\":{\"paddingBottom\":\"83.2%\",\"position\":\"relative\",\"bottom\":\"0\",\"left\":\"0\",\"backgroundImage\":\"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAARCAYAAADdRIy+AAAACXBIWXMAABYlAAAWJQFJUiTwAAADj0lEQVQ4y3WU709bZRTHL/1929v29raX9va2vbft7Q8KdMAAhyJhis3GYGSEBOfwhYsuuqlb0BkdxqlzYyoanNnIsgDbkLiJb+YkapboG+Ob+S+Y+Mp/42NukVKmvjh5nifnnO/5PueX4I4YbItLztD83iXybpvt0xFKE04WSeQ7ECQd4f+Amh2bpdnGG80iOKIcmpri9p0lHCED4VGjZkePYuKLZhHV/L/0ttQBWxTGpqe5efc6gk/bYWjT9UZNfLFs3dCn5hCkFKfPnaE6MNTQbwdqZlibmuTq+he0SOkthp6IQbrcvRXBr+MMpRE8CYaPHOLe77eZenkGIZDC35pvBKyzj9mAMYYnx5lf/QjBn0KwowrOKK+88RqLKwt07x8m3zfA6EszzG1cYfbbJWa+vERQL+FVzDqoLaKaY8tXpf/wGLNLF3AE/gFUzSqZSi8P//qJ7/7c5L2f7/Dqj98wubbKk9dWGLyxjjExg0fSEFutRl69sRyCK0bl4DjHPr+Ex/6ZGMsR1IoILpWjb59i8Y9fOfrgAY9//QN7rm5gza+T/mCd1DvLiFYfoqwTSBTr33YrJoI7hlmbYOTiZTx226hmJ6nSXiJ6G0q+k2P37zF4/zespV/IvP891utrpI/fJP3iVySfW0TK7cMb1vAqGTx2UTwx4k9N0Ht+ASmSQZD1MrmOgXpUtz+B9fwpqncfknp3k+zxDUpHVqkcuI719A2s8TVyUytoB+fwm3245DQOn4o8OEHb2c/wBTUEu1oxowObqVvOEEhYGHO3yJ7epHBgmcLANcq986T3XCbX/SnF4RWM0St44qV66zhFFan/MJkTHyOGtK22aQnqBOIWIb1MQEkj99TIntigvH8Zs+sCRtcZ+gdeINF+llzPJ4T0QZyhOG4lh9OvInWOYk5/WM+v0BgjxaTQNYhe6kNU0kR6JimM3SLXu0C26yQjw8+iVd8ilK6hmm0YlX04wxmcUivBYo1s7Ry+SBOgrbRbwWx/DMnOZ1hHbnuGzOB5zP6LqO1vEtCGkPUi+eoTSPECLtmoMw0ZI5j9s3giyZ1Z3j6DWgnN6iasV+qV9MfzBPUuxJiFkiqRyHcTTpZxhFK4IybOcAJJG8KsnkRUMzsM6yOomI2FkGnrQ9JKiGoBj2IQzVRIFvY27Bp+4SRivJewMY5LTu5eX49uErvh7budCjsN/7XGtu42kVx9yfwNFqH+N9XmvZIAAAAASUVORK5CYII=')\",\"backgroundSize\":\"cover\",\"display\":\"block\"}}),`\n `,mdx(\"picture\",{parentName:\"span\"},`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/ee6227e201854f01e66247f7ce73b036/601b1/lassoCDHist.webp 500w\",\"/static/ee6227e201854f01e66247f7ce73b036/64775/lassoCDHist.webp 984w\"],\"sizes\":\"(max-width: 984px) 100vw, 984px\",\"type\":\"image/webp\"}),`\n `,mdx(\"source\",{parentName:\"picture\",\"srcSet\":[\"/static/ee6227e201854f01e66247f7ce73b036/0eb09/lassoCDHist.png 500w\",\"/static/ee6227e201854f01e66247f7ce73b036/61b91/lassoCDHist.png 984w\"],\"sizes\":\"(max-width: 984px) 100vw, 984px\",\"type\":\"image/png\"}),`\n `,mdx(\"img\",{parentName:\"picture\",\"className\":\"gatsby-resp-image-image\",\"src\":\"/static/ee6227e201854f01e66247f7ce73b036/61b91/lassoCDHist.png\",\"alt\":\"Coordinate Descent for lasso\",\"title\":\"Coordinate Descent for lasso\",\"loading\":\"lazy\",\"decoding\":\"async\",\"style\":{\"width\":\"100%\",\"height\":\"100%\",\"margin\":\"0\",\"verticalAlign\":\"middle\",\"position\":\"absolute\",\"top\":\"0\",\"left\":\"0\"}}),`\n `),`\n `))),mdx(\"p\",null,`Ok, that was 1. incredibly quick and 2. somewhat `,mdx(\"em\",{parentName:\"p\"},`zigzaggy`),`, as expected. You can see how the first step\nonly adjusts b and the second step only adjusts m (you might have to zoom in a bit to see it more clearly).\nThen there are some tiny improvements but\nafter the first steps the minimum is already found. But why did it converge so quickly?!\nThis makes subgradient descent look a lot worse when compared directly to coordinate descent.\nSo what is the reason behind the speed of coordinate descent? If you want to find out how coordinate descent works,\nhow you can implement it, and most importantly, why it is so extremely fast,\nyou can take a deeper dive and read `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_math/coordinate_descent\"},`Coordinate Descent Explained, Step by Step`),`.\nIn that article, you will learn everything about how coordinate descent works, how you can implement it from scratch\nand why it is so incredibly fast.\nSo Coordinate descent is basically as good as it gets.`),mdx(\"p\",null,`But why did we then even look at subgradient descent if, in practice,\nyou’ll basically always use coordinate descent?\nThe reason is that subgradient descent is quite a bit easier to implement,\nand thus its implementation is a lot less prone to errors than the one of coordinate descent.\nThis is also the reason why in almost every book and every “machine learning from scratch”-course,\nyou will be presented with `,mdx(\"em\",{parentName:\"p\"},`just`),` an implementation of subgradient descent for lasso.\nOnly sometimes is coordinate descent mentioned and even rarer is it actually implemented.\nThis is why I have created the articles `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_math/subgradient_descent\"},`Subgradient Descent Explained, Step by Step`),`\nas well as `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_math/coordinate_descent\"},`Coordinate Descent Explained, Step by Step`),`.\nBy reading these articles, you will understand exactly where the limits of subgradient descent are and\nhow coordinate descent is able to perform so well. In these articles, we will also implement the mentioned algorithms\nfrom scratch, so you can truly understand how these variations of gradient descent work under the hood.\nI highly recommend that you give these two a read\nafter finishing this article.`),mdx(\"p\",null,`Now I don’t want to let you go without showing you some practical code,\nso let’s look at how we can perform lasso regression with scikit-learn (which uses coordinate descent\nunder the hood!).`),mdx(\"h2\",{\"id\":\"implementing-lasso-using-scikit-learn\"},`Implementing Lasso using Scikit-Learn`),mdx(\"p\",null,`Using lasso with scikit-learn is quite simple with the `,mdx(\"a\",{parentName:\"p\",\"href\":\"https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html\",\"target\":\"_blank\",\"rel\":\"nofollow noopener noreferrer\"},mdx(\"inlineCode\",{parentName:\"a\"},`Lasso`)),`-class.`),mdx(\"p\",null,`But before we can perform our regression it is `,mdx(\"em\",{parentName:\"p\"},`crucial`),` that we standardize our data,\nbecause ridge and lasso are sensitive to the scaling of the data.\nIf you’re interested in what can go wrong if you don’t standardize your data\nbefore using ridge and lasso, and how exactly standardization solves this problem,\ncheck out the article `,mdx(\"a\",{parentName:\"p\",\"href\":\"/dataset_optimization/standardization\"},`When You Should Standardize Your Data`),`.`),mdx(\"p\",null,`Here’s our data:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`X = np.array([1.25, 1. , 0.75, 1.5, 1.75, 1.5 , 0.75])\n# since sklearn expects a 2d-array for our X,\n# we add an additional dimension along the first axis using the following command\nX = np.expand_dims(X,1)\ny = np.array([40. , 42. , 46. , 37., 40. , 38. , 39.8])\n`)),mdx(\"p\",null,`The first thing we do is we split our dataset into a training and testing portion:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`X_train, X_test = X_s[:4], X_s[4:]\ny_train, y_test = y[:4], y[4:]\n`)),mdx(\"p\",null,`Then we create a `,mdx(\"inlineCode\",{parentName:\"p\"},`StandardScaler`),` and train it using only our\ntraining dataset. All this “training” does is calculate the mean\nand standard deviation of our dataset and store them.\nThen, we standardize our X:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`from sklearn.preprocessing import StandardScaler\n\nscaler = StandardScaler()\nscaler.fit(X_train)\nX_s_train = scaler.transform(X_train)\nX_s_test = scaler.transform(X_test)\n`)),mdx(WarningTip,{mdxType:\"WarningTip\"},mdx(\"p\",null,`When you transform your data in any way (f.e. standardization),\nyou should always only use your training set to compute the\nvariables necessary for the transformation, i.e. in our example you should only use the\ntraining dataset to fit the `,mdx(\"inlineCode\",{parentName:\"p\"},`Standardscaler`),`. Then, you also always have to use the\nsame transformer (in our example the same `,mdx(\"inlineCode\",{parentName:\"p\"},`StandardScaler`),`-object) to transform all of your data,\ntraining and testing.`),mdx(\"p\",null,`This can be a bit challenging to understand and memorize at first, and when I first heard about this,\nI asked myself why these rules are so important. A lot of machine learning resources don’t actually\ntalk about why this is the case, but I think it is very important to understand this well.\nFor this reason I have created the article `,mdx(\"a\",{parentName:\"p\",\"href\":\"/dataset_optimization/standardization\"},`When You Should Standardize Your Data`),`,\nwhere I explain step-by-step how this entire process of standardization works and why you have\nto follow these somewhat specific rules I just mentioned. If you want to take a deeper dive\ninto standardization, I highly recommend you give the article a read!`)),mdx(\"p\",null,`And now we can perform our regression as follows:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`from sklearn.linear_model import Lasso\n\nlasso = Lasso()\nlasso.fit(X_s_train, y_train)\nprint(lasso.coef_[0], lasso.intercept_)\n# prints: -2.5269426990176953 40.63053220519125\n`)),mdx(\"p\",null,`Instead of manually standardizing our data before feeding it into\nour `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),` model, we can also use a so-called `,mdx(\"em\",{parentName:\"p\"},`pipeline`),` to dynamically\nstandardize the data that goes into our model. This way we don’t have to duplicate\nour dataset in memory, which saves space. This approach would look like this in code:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`pipeline = make_pipeline(StandardScaler(), Lasso())\npipeline.fit(X_train, y_train)\nprint(pipeline[1].coef_[0], pipeline[1].intercept_)\n# prints: -2.5269426990176953 40.63053220519125\n`)),mdx(HelpfulTip,{mdxType:\"HelpfulTip\"},mdx(\"p\",null,`As you can see, the results are exactly the same! `,mdx(\"inlineCode\",{parentName:\"p\"},`make_pipeline`),` saves us a few lines\nof code and reduces the potential of error, because it automatically only uses the training\ndata to create our `,mdx(\"inlineCode\",{parentName:\"p\"},`StandardScaler`),` and it also always uses the same `,mdx(\"inlineCode\",{parentName:\"p\"},`StandardScaler`),`-object\nto transform our data, which is something we would have to take care of manually, if we were\nto not use a pipeline. Also, we don’t have to duplicate the dataset in memory.\nBecause of these reasons I would recommend you use these pipelines\nin practical projects, instead of doing the processing manually.`)),mdx(\"h3\",{\"id\":\"parameter-sparsity-testing-for-lasso\"},`Parameter Sparsity Testing for Lasso`),mdx(\"p\",null,`Earlier we’ve talked about how lasso regression has the ability to produce\nsparse model parameters. To make sure this is actually the case, we can perform a simple test.\nWe create a completely random X, a completely random y, and then we run lasso on it.\nIdeally, every single parameter should be set to zero because there should be no correlation\nbetween our X and our y. Let’s try it out:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`X_rand = np.random.rand(100,50)\ny_rand = np.random.rand(100)\n\npipeline = make_pipeline(StandardScaler(), Lasso())\n\npipeline.fit(X_rand, y_rand)\n\nprint(pipeline[1].coef_, pipeline[1].intercept_)\n# output:\n# [-0. 0. -0. 0. -0. 0. -0. -0. -0. 0. -0. 0. 0. 0. -0. -0. -0. 0.\n# 0. 0. -0. -0. -0. -0. 0. 0. 0. -0. -0. 0. -0. 0. 0. 0. -0. 0.\n# 0. -0. 0. -0. 0. -0. 0. -0. 0. 0. -0. 0. -0. 0.] 0.48078044760264155\n`)),mdx(\"p\",null,`As we can see, every weight has been zeroed out. The only exception to this is the bias (or intercept) of our lasso\nmodel. This is because scikit-learn’s lasso implementation does not regularize the bias.`),mdx(\"h3\",{\"id\":\"lassos-lesser-known-twin-sgdregressor\"},mdx(\"inlineCode\",{parentName:\"h3\"},`Lasso`),`’s Lesser-Known Twin: `,mdx(\"inlineCode\",{parentName:\"h3\"},`SGDRegressor`)),mdx(\"p\",null,`Apart from `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),`, scikit-learn also provides a another class that can be used to create\na lasso model, namely `,mdx(\"a\",{parentName:\"p\",\"href\":\"https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDRegressor.html\",\"target\":\"_blank\",\"rel\":\"nofollow noopener noreferrer\"},mdx(\"inlineCode\",{parentName:\"a\"},`SGDRegressor`)),`\nwhich, as the name suggests, uses stochastic gradient Ddscent (SGD). Maybe you’re scratching\nyour head right now. Didn’t we state that the L1 penalty is not suited for regular (stochastic) gradient descent\nbecause the absolute values are not differentiable at 0?\nYes, we did. However, there are a number of methods to alter the penalty and make it suitable again.\nSpecifically, we mentioned subgradient descent and coordinate descent.\nBut there are even more options to choose from!\nScikit-learn for example uses a `,mdx(\"em\",{parentName:\"p\"},`truncated`),` gradient instead of a regular gradient for its `,mdx(\"inlineCode\",{parentName:\"p\"},`SGDRegressor`),`-class.`),mdx(\"p\",null,`This approach is similar to regular gradient descent, but the difference is that the weight updating process is divided\ninto two steps. The first step is updating the weight without the L1 penalty.\nThe second step is applying the L1 penalty. The trick here is that the penalty is only applied to such an extent\nas to not change the sign of the weight. In other words, the weight is cut off (or `,mdx(\"em\",{parentName:\"p\"},`truncated`),`) at zero.\nThis happens for every weight and thus we have a `,mdx(\"em\",{parentName:\"p\"},`truncated`),` gradient instead of a regular one.\nYou can read more about this particular approach in the paper by Tsuruoka et al.\ntitled “Stochastic Gradient Descent Training for L1-regularized Log-linear Models with Cumulative Penalty”. `,mdx(ToolTip,{mdxType:\"ToolTip\"},`If you’re interested, you can read it on ResarchGate `,mdx(\"a\",{parentName:\"p\",\"href\":\"https://www.researchgate.net/publication/220874186_Stochastic_Gradient_Descent_Training_for_L1-regularized_Log-linear_Models_with_Cumulative_Penalty\",\"target\":\"_blank\",\"rel\":\"nofollow noopener noreferrer\"},`here`),`.`)),mdx(\"p\",null,`Let’s try it out:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`pipeline_sgd = make_pipeline(StandardScaler(), SGDRegressor(alpha=1, penalty=\"l1\"))\npipeline_sgd.fit(X_train, y_train)\nprint(pipeline_sgd[1].coef_[0], pipeline_sgd[1].intercept_)\n# prints: -2.2233194459768386 [40.87110655]\n`)),mdx(\"p\",null,`Nice, it produces almost the same result as our `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),` model.\nWe can also test if it produces sparse weights by fitting a `,mdx(\"inlineCode\",{parentName:\"p\"},`SGDRegressor`),` model on our random dataset:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`pipeline_sgd = make_pipeline(StandardScaler(), SGDRegressor(alpha=1, penalty=\"l1\"))\npipeline_sgd.fit(X_rand, y_rand)\nprint(pipeline_sgd[1].coef_, pipeline_sgd[1].intercept_)\n# output:\n# [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n# 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n# 0. 0.] [0.46368558]\n`)),mdx(\"p\",null,`Nice! In general, coordinate descent will converge quite a bit faster than this truncated SGD,\nso in practice you should usually stick to using `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),`.`),mdx(\"h2\",{\"id\":\"finding-the-optimal-value-for-alpha\"},`Finding the Optimal Value for `,mdx(\"span\",{parentName:\"h2\",\"className\":\"math math-inline\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex\"},mdx(\"span\",{parentName:\"span\",\"className\":\"katex-mathml\"},mdx(\"math\",{parentName:\"span\",\"xmlns\":\"http://www.w3.org/1998/Math/MathML\"},mdx(\"semantics\",{parentName:\"math\"},mdx(\"mrow\",{parentName:\"semantics\"},mdx(\"mi\",{parentName:\"mrow\"},`α`)),mdx(\"annotation\",{parentName:\"semantics\",\"encoding\":\"application/x-tex\"},`\\\\alpha`)))),mdx(\"span\",{parentName:\"span\",\"className\":\"katex-html\",\"aria-hidden\":\"true\"},mdx(\"span\",{parentName:\"span\",\"className\":\"base\"},mdx(\"span\",{parentName:\"span\",\"className\":\"strut\",\"style\":{\"height\":\"0.43056em\",\"verticalAlign\":\"0em\"}}),mdx(\"span\",{parentName:\"span\",\"className\":\"mord mathnormal\",\"style\":{\"marginRight\":\"0.0037em\"}},`α`)))))),mdx(\"p\",null,`For ridge regression, scikit-learn offers `,mdx(\"inlineCode\",{parentName:\"p\"},`RidgeCV`),`.\nFor lasso regression, there is `,mdx(\"inlineCode\",{parentName:\"p\"},`LassoCV`),`!\nIt works in exactly the same way as `,mdx(\"inlineCode\",{parentName:\"p\"},`RidgeCV`),`, which we’ve explored `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression/#finding-the-optimal-value-for-alpha\"},`here`),`.\nSince our dataset is so small,\n`,mdx(\"inlineCode\",{parentName:\"p\"},`LassoCV`),` will throw an error if we try to fit it using our current `,mdx(\"inlineCode\",{parentName:\"p\"},`X_train`),` and `,mdx(\"inlineCode\",{parentName:\"p\"},`y_train`),`.\nWhat we can do is increase the size of our training set:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`X_train, X_test = X_s[:5], X_s[5:]\ny_train, y_test = y[:5], y[5:]\n`)),mdx(\"p\",null,`Now we can use `,mdx(\"inlineCode\",{parentName:\"p\"},`LassoCV`),` like that:`),mdx(\"pre\",null,mdx(\"code\",{parentName:\"pre\",\"className\":\"language-python\"},`lasso_cv_pipeline = make_pipeline(StandardScaler(),\n LassoCV(alphas=[0.1, 1.0, 10]))\nlasso_cv_pipeline.fit(X_train, y_train)\nprint(lasso_cv_pipeline[1].alpha_)\n# output: 0.1\n`)),mdx(\"h2\",{\"id\":\"further-reading\"},`Further Reading`),mdx(\"p\",null,`Good job on finishing this article!\nIf you want to continue learning, here are some recommendations.`),mdx(\"h3\",{\"id\":\"ridge-lasso-or-both\"},`Ridge, Lasso, or… both?`),mdx(\"p\",null,`In this article, we took a look at lasso. In the `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/ridge_regression/\"},`previous one`),`, we took a look at ridge.\nSo which one should you choose?\nWell… I’ve never said that you have to make the choice, did I?\nIn fact, you can use `,mdx(\"strong\",{parentName:\"p\"},`both`),` ridge and lasso at the same time!\nHow? We can use something that’s called `,mdx(\"em\",{parentName:\"p\"},`elastic net`),`.\nIf you’re interested as to how this works,\ntake a look at `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/elastic_net_regression\"},`Elastic Net Explained, Step by Step`),`.\nThe article also has a nice summary that goes over all of the scikit-learn-classes like\n`,mdx(\"inlineCode\",{parentName:\"p\"},`Ridge`),` or `,mdx(\"inlineCode\",{parentName:\"p\"},`SGDRegressor`),` and explains them in an organized fashion!`),mdx(\"h3\",{\"id\":\"standardization\"},`Standardization`),mdx(\"p\",null,`In the article about ridge regression, we saw that not standardizing our data can lead to some\nproblems. In this article, we’ve standardized our data before fitting our `,mdx(\"inlineCode\",{parentName:\"p\"},`Lasso`),` model.\nBut how exactly does standardization work and why is it so important to use when dealing with\nregularized models? If you want to read some clear and easy-to-understand answers to those questions,\nhead right on to the article `,mdx(\"a\",{parentName:\"p\",\"href\":\"/dataset_optimization/standardization\"},`When You Should Standardize Your Data`),` to find out more!`),mdx(\"h3\",{\"id\":\"lasso-penalty-for-other-models\"},`Lasso Penalty for Other Models`),mdx(\"p\",null,`In this article, we’ve looked at lasso regression, which is an adaptation of OLS regression.\nBut like with ridge, we can also take the lasso penalty and\napply it to other models, like logistic regression or polynomial regression! If you’re interested in these regularized models,\nI recommend you take a look at the articles `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/logistic_regression\"},`Logistic Regression Explained, Step by Step`),`\nand `,mdx(\"a\",{parentName:\"p\",\"href\":\"/machine_learning_models/polynomial_regression\"},`Polynomial Regression Explained, Step by Step`),`.\nThere you will learn everything you need about the named models as well as their regularized variants!`));};MDXContent.isMDXComponent=true;","excerpt":"Lasso regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. Lasso regression is very similar to ridge regression, but there are some key differences between the two that you will have to understand if you want to use them effectively. In this article, you will learn everything you need to know about lasso regression, the differences between lasso and ridge, as well as how you can start using lasso regression in your own machine learning projects.","tableOfContents":{"items":[{"url":"#outline","title":"Outline"},{"url":"#prerequisites","title":"Prerequisites"},{"url":"#the-problem","title":"The Problem"},{"url":"#the-qualitative-difference-between-ridge-and-lasso","title":"The Qualitative Difference Between Ridge and Lasso","items":[{"url":"#parameter-sparsity-of-lasso","title":"Parameter Sparsity of Lasso"}]},{"url":"#solving-lasso-regression","title":"Solving Lasso Regression","items":[{"url":"#visualizing-subgradient-descent-and-coordinate-descent","title":"Visualizing Subgradient Descent and Coordinate Descent"}]},{"url":"#implementing-lasso-using-scikit-learn","title":"Implementing Lasso using Scikit-Learn","items":[{"url":"#parameter-sparsity-testing-for-lasso","title":"Parameter Sparsity Testing for Lasso"},{"url":"#lassos-lesser-known-twin-sgdregressor","title":"Lasso's Lesser-Known Twin: SGDRegressor"}]},{"url":"#finding-the-optimal-value-for-alpha","title":"Finding the Optimal Value for \\alpha"},{"url":"#further-reading","title":"Further Reading","items":[{"url":"#ridge-lasso-or-both","title":"Ridge, Lasso, or... both?"},{"url":"#standardization","title":"Standardization"},{"url":"#lasso-penalty-for-other-models","title":"Lasso Penalty for Other Models"}]}]},"frontmatter":{"title":"Lasso Regression Explained, Step by Step","userDate":"20 June 2021","date":"2021-06-20T16:35:36.458Z","dateModified":"2021-06-26T08:28:02.234Z","changelog":[["2021-06-26T08:28:02.234Z","Added section about SGDRegressor."],["2022-01-18T09:43:47.766Z","Fixed small typo in section about qualitative difference between Ridge and Lasso. Thanks to Tianyu for pointing this out!"]],"tags":["Machine Learning Models"],"timeToRead":16,"excerpt":"Lasso regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. Lasso regression is very similar to ridge regression, but there are some key differences between the two that you will have to understand if you want to use them effectively. In this article, you will learn everything you need to know about lasso regression, the differences between lasso and ridge, as well as how you can start using lasso regression in your own machine learning projects.","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACQElEQVQozz3OSXOaYADGcT5CzC1NqqzyIjsCyssSIyIKiIJLQNTapnGJTZpMD520nUw+eoceOvO7PYfnj0z77t0ivbtNd/k8C/tFPFyOhsu4VIyj1SRaTUalZFRMomIcribRMg6zcJBHAZKF/mF1+/x1+2P/ZZfPdsv5YZ3vi8Vxszxuiv/us+lxnc2DwaTv3uezcl0vkYHVmg+90Y21moSzQXfa7ywCL/Wus2hQxMG/hKCIg6Rrp72OB9tOU0n93maWJL6HoBdXtiI5Mu+bLZNvOBJ/3RRcTYEcMDna5gHkaFtkHVmEAtdiGc+CrtF2oaEwAKlUPrCAt5tSNryxBRoyeGxr9+v8bpX/ef35bb97eXo8bNeRqUZQUSjMksU+1Ia2YUs8cn5+geMNHG/okqyxILT0DkdFN/Yk8JfpuEii9TRehP2uxHhNTq/jjio7qtLRVQXUkUrlgiS5OsVdXhLRKHl/e3v//fpyOj49HJ5Ph912s//86fF4ePv1+v3hEPe6hsB1WpqjqQJFIGdn5TNJNEiSpUhWU40gHJ9Oj3k6TgM/HvTn4ygNh1ky9juOIQmmXIKyyKA1hCDYVvtaktqCoOE4g+NM9SOFYQCQjMoyjsxZImuKrME3TIk3RN6FRpOhWwJXv7pEyCql1FmG5lGUJgmGIssEkmgQRBnS5ITA1JOOEVr6EOqmyHm26Tum2ZQYrIqogFWBQOFMDaVrKI2VAIYBHAMYClAUUEQDyrKrK11daQtszzI9C0JZArXqX0zno6oMKKD7AAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/1cbee7448495e7e7fa3771995e853206/87706/LassoRegression_Thumbnail.png","srcSet":"/static/1cbee7448495e7e7fa3771995e853206/0dee1/LassoRegression_Thumbnail.png 750w,\n/static/1cbee7448495e7e7fa3771995e853206/8beaa/LassoRegression_Thumbnail.png 1080w,\n/static/1cbee7448495e7e7fa3771995e853206/87706/LassoRegression_Thumbnail.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/1cbee7448495e7e7fa3771995e853206/a66aa/LassoRegression_Thumbnail.webp 750w,\n/static/1cbee7448495e7e7fa3771995e853206/65dd5/LassoRegression_Thumbnail.webp 1080w,\n/static/1cbee7448495e7e7fa3771995e853206/71d4d/LassoRegression_Thumbnail.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Lari Giba","bio":"Hi! I'm Lari and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAbABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAMEAgH/xAAWAQEBAQAAAAAAAAAAAAAAAAABAgP/2gAMAwEAAhADEAAAAeDIop5sDKn0aCSgzv8A/8QAHhABAAICAQUAAAAAAAAAAAAAAQACAxEEEyEiMjP/2gAIAQEAAQUC3ZAE6ZGW8HtMw1M2O+tTlfI9Z//EABURAQEAAAAAAAAAAAAAAAAAABAR/9oACAEDAQE/AYU//8QAFxEBAAMAAAAAAAAAAAAAAAAAABAREv/aAAgBAgEBPwFbMf/EABwQAAIBBQEAAAAAAAAAAAAAAAABMQIQESFxkf/aAAgBAQAGPwLfpJvBhuRVIgXRNwpsxct//8QAHRABAAMAAgMBAAAAAAAAAAAAAQARIUFRMWGBcf/aAAgBAQABPyG78vaUcDUeYJrJrCUcHAhzK6pYRVinqYAdKT7Y0/Yi+8UGE//aAAwDAQACAAMAAAAQsO1C/8QAGBEBAQEBAQAAAAAAAAAAAAAAAREAECH/2gAIAQMBAT8QPF4uTBS7/8QAGBEBAQEBAQAAAAAAAAAAAAAAAQARIVH/2gAIAQIBAT8QO2vYEgOZf//EAB8QAQEAAgIBBQAAAAAAAAAAAAERACExQVFhcZGxwf/aAAgBAQABPxDlRMDhDreGljrJ8uK5DiW/WJE4rXSx7wgAOy8sEKuzsxERAF59XxirS0k0dz03g4T3iY5jH9MRu2FvejEgEAAM/9k=","aspectRatio":0.7407407407407407,"src":"/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg","srcSet":"/static/9eab840ebd4087faf31c48702c92af36/b3d27/lari.jpg 40w,\n/static/9eab840ebd4087faf31c48702c92af36/5a735/lari.jpg 80w,\n/static/9eab840ebd4087faf31c48702c92af36/6e81a/lari.jpg 120w,\n/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg 800w,\n/static/9eab840ebd4087faf31c48702c92af36/75b83/lari.jpg 1520w","sizes":"(max-width: 800px) 100vw, 800px"}}]}}],"imgCaption":"Background image by Nick Tiemeyer","imgSource":"https://unsplash.com/photos/i9cldzefwJY","uniquePostId":10,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner"}},"relatedPosts":{"totalCount":4,"edges":[{"node":{"id":"8da3beae-1c9f-5fae-a6b9-3fd103f104ef","excerpt":"Elastic net is a combination of the two most popular regularized variants of linear regression: ridge and lasso. Ridge utilizes an L2 penalty and lasso uses an L1 penalty. With elastic net, you don't have to choose between these two models, because elastic net uses both the L2 and the L1 penalty! In practice, you will almost always want to use elastic net over ridge or lasso, and in this article you will learn everything you need to know to do so, successfully!","frontmatter":{"title":"Elastic Net Regression Explained, Step by Step","timeToRead":7,"date":"2021-06-26T08:28:02.234Z"},"fields":{"slug":"/machine_learning_models/elastic_net_regression/"}}},{"node":{"id":"f6288123-30a0-5918-8d58-1607badab75e","excerpt":"Lasso regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. Lasso regression is very similar to ridge regression, but there are some key differences between the two that you will have to understand if you want to use them effectively. In this article, you will learn everything you need to know about lasso regression, the differences between lasso and ridge, as well as how you can start using lasso regression in your own machine learning projects.","frontmatter":{"title":"Lasso Regression Explained, Step by Step","timeToRead":16,"date":"2021-06-20T16:35:36.458Z"},"fields":{"slug":"/machine_learning_models/lasso_regression/"}}},{"node":{"id":"a9afab9b-71b2-5906-a280-fb2dd2afa180","excerpt":"Ridge Regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. In this article, you will learn everything you need to know about Ridge Regression, and how you can start using it in your own machine learning projects.","frontmatter":{"title":"Ridge Regression Explained, Step by Step","timeToRead":26,"date":"2021-05-23T19:13:45.907Z"},"fields":{"slug":"/machine_learning_models/ridge_regression/"}}},{"node":{"id":"6be475fd-5ce4-55fb-a78f-49f61e026a2e","excerpt":"Linear regression is one of the most famous algorithms in statistics and machine learning. In this post you will learn how linear regression works on a fundamental level. You will also implement linear regression both from scratch as well as with the popular library scikit-learn in Python. You will learn when and how to best use linear regression in your machine learning projects. You do not need any knowledge prior to reading this article.","frontmatter":{"title":"Linear Regression Explained, Step by Step","timeToRead":23,"date":"2021-03-20T18:17:30.352Z"},"fields":{"slug":"/machine_learning_models/linear_regression/"}}}]}},"pageContext":{"slug":"/machine_learning_models/lasso_regression/","prev":{"excerpt":"Ridge Regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. In this article, you will learn everything you need to know about Ridge Regression, and how you can start using it in your own machine learning projects.","frontmatter":{"title":"Ridge Regression Explained, Step by Step","tags":["Machine Learning Models"],"date":"2021-05-23T19:13:45.907Z","dateModified":"2021-11-02T08:08:00.157Z","changelog":[["2023-01-17T20:08:21.833Z","Fixed a typo. Thanks to ag for pointing this out!"],["2023-01-17T18:50:31.151Z","Fixed a typo. Thanks to Андрей for pointing this out!"],["2021-11-02T08:08:00.157Z","Added 1/n to the definition of the MSE which I previously forgot in one section. Thanks to @ML_Schlagy for pointing this out!"]],"draft":false,"excerpt":"Ridge Regression is an adaptation of the popular and widely used linear regression algorithm. It enhances regular linear regression by slightly changing its cost function, which results in less overfit models. In this article, you will learn everything you need to know about Ridge Regression, and how you can start using it in your own machine learning projects.","uniquePostId":9,"timeToRead":26,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACVUlEQVQozz3OWU8aUQCG4QMeBmEYhmFmmOWcWZgVDqACKkOCCzZao8SaaohCG5qmS2wv3JDlptg7TZo2aYzav+CvbDBNk+fuTb58gEopkZT6H0ypEU6lOJVKKRSnTCsr/5PMQEaEjAATAkzwkOZAVDApwYw8g1NGmNNCLAIsAowcSkghZiqclCGnQlZ+Jj2v8CAuWrMoH8UkigiFCRCsfK3Z+3hSXXlRCtYbWy3VLyN/QbAKIU6DvA7TGkxjOP0rA4bDjGgljXnaWqLdAGjlne6nu4fHi8H41++7u/v72x8//zw+tHsfQBLHNEIpHpRsKJpxxQJ0Un6m6M6cahcjvMFiz1sI3n8+ebm1ubHeHAxHJGialdUZXAzLPpScWcWlkSuYeRCNC1FaiMb5fn9w1b/qdrqt1t75ZX88HnWOj990upubW68O2tu7e+3O24Oj7v5Bm4oxaVHJOj6QkSMjR8Xul6+nw9H42/X3m5vbp6ens7PT4WAwHI0uLy+uJ5OL8/PJZDIcjXq9dzutvf3Xh43VJpAUS1KyCnL4jIF0L19YrC43avVVrDmW5WCMi8VSsTRHSCEI6mvrG4fto8Va3fGJpGLASC5tVChaTPHIcudJaTnrFCXF1owc1n0j69m2l8uTSqVaLleCel1BGgCh0Aw1Q8VAdnnHqe3auXJ1aaWxtj1XDkybYMMXJENSbd3MmxZxPOJ6Oc/PIayzXDrBpqgYE5lNAE62OcnK2iRXWPRJFelekpOFjDFfrvOioSLHsIiCXd30sZ7NKIjjRU4QYwk2TMX/AgxCcEm95chYAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/bffad91bca399e12f65048348b2b2df8/87706/RidgeRegression_Thumbnail.png","srcSet":"/static/bffad91bca399e12f65048348b2b2df8/0dee1/RidgeRegression_Thumbnail.png 750w,\n/static/bffad91bca399e12f65048348b2b2df8/8beaa/RidgeRegression_Thumbnail.png 1080w,\n/static/bffad91bca399e12f65048348b2b2df8/87706/RidgeRegression_Thumbnail.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/bffad91bca399e12f65048348b2b2df8/a66aa/RidgeRegression_Thumbnail.webp 750w,\n/static/bffad91bca399e12f65048348b2b2df8/65dd5/RidgeRegression_Thumbnail.webp 1080w,\n/static/bffad91bca399e12f65048348b2b2df8/71d4d/RidgeRegression_Thumbnail.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Lari Giba","bio":"Hi! I'm Lari and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"aspectRatio":0.7434944237918215,"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAbABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAMEAgH/xAAWAQEBAQAAAAAAAAAAAAAAAAABAgP/2gAMAwEAAhADEAAAAeDIop5sDKn0aCSgzv8A/8QAHhABAAICAQUAAAAAAAAAAAAAAQACAxEEEyEiMjP/2gAIAQEAAQUC3ZAE6ZGW8HtMw1M2O+tTlfI9Z//EABURAQEAAAAAAAAAAAAAAAAAABAR/9oACAEDAQE/AYU//8QAFxEBAAMAAAAAAAAAAAAAAAAAABAREv/aAAgBAgEBPwFbMf/EABwQAAIBBQEAAAAAAAAAAAAAAAABMQIQESFxkf/aAAgBAQAGPwLfpJvBhuRVIgXRNwpsxct//8QAHRABAAMAAgMBAAAAAAAAAAAAAQARIUFRMWGBcf/aAAgBAQABPyG78vaUcDUeYJrJrCUcHAhzK6pYRVinqYAdKT7Y0/Yi+8UGE//aAAwDAQACAAMAAAAQsO1C/8QAGBEBAQEBAQAAAAAAAAAAAAAAAREAECH/2gAIAQMBAT8QPF4uTBS7/8QAGBEBAQEBAQAAAAAAAAAAAAAAAQARIVH/2gAIAQIBAT8QO2vYEgOZf//EAB8QAQEAAgIBBQAAAAAAAAAAAAERACExQVFhcZGxwf/aAAgBAQABPxDlRMDhDreGljrJ8uK5DiW/WJE4rXSx7wgAOy8sEKuzsxERAF59XxirS0k0dz03g4T3iY5jH9MRu2FvejEgEAAM/9k=","sizes":"(max-width: 800px) 100vw, 800px","src":"/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg","srcSet":"/static/9eab840ebd4087faf31c48702c92af36/fd013/lari.jpg 200w,\n/static/9eab840ebd4087faf31c48702c92af36/25252/lari.jpg 400w,\n/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg 800w,\n/static/9eab840ebd4087faf31c48702c92af36/0ff54/lari.jpg 1200w,\n/static/9eab840ebd4087faf31c48702c92af36/75b83/lari.jpg 1520w"}}]}}]},"fields":{"layout":"post","slug":"/machine_learning_models/ridge_regression/"}},"next":{"excerpt":"Elastic net is a combination of the two most popular regularized variants of linear regression: ridge and lasso. Ridge utilizes an L2 penalty and lasso uses an L1 penalty. With elastic net, you don't have to choose between these two models, because elastic net uses both the L2 and the L1 penalty! In practice, you will almost always want to use elastic net over ridge or lasso, and in this article you will learn everything you need to know to do so, successfully!","frontmatter":{"title":"Elastic Net Regression Explained, Step by Step","tags":["Machine Learning Models"],"date":"2021-06-26T08:28:02.234Z","dateModified":"2021-06-26T08:28:02.234Z","changelog":[],"draft":false,"excerpt":"Elastic net is a combination of the two most popular regularized variants of linear regression: ridge and lasso. Ridge utilizes an L2 penalty and lasso uses an L1 penalty. With elastic net, you don't have to choose between these two models, because elastic net uses both the L2 and the L1 penalty! In practice, you will almost always want to use elastic net over ridge or lasso, and in this article you will learn everything you need to know to do so, successfully!","uniquePostId":11,"timeToRead":7,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACWElEQVQozxXNW0/TUAAA4MYYQ0A2oGVr19tO29Oedr3fd7p1WwtjbBMRjItBBgIGUGQ8aZCor/iq/mDj9wc+oml4optt8Eq1ztegs86Cap2jgWakpVfua0kh2rGeFLgYDcb7e7P5u/Orw7fzOC9Xaxzh4r7XGcotT7F8aHnIjZATa06EvNgIcbq9l48P28XYxgWKeyjI6gBJLR+YQZVTCA/3zBAzss7CFgcNVkHQTVQ3kU1fNr2GpDHQUdJdwelUGmCdlYHu/G98XGGaREMy9KCtOqGIrCaykrwALW+NldZZwGn2KtMkBdXIdmtqQLKA4hUeOaJmaU5UAzpB0ZKMXKA7myIMs973Hz8/LxaX1x+PT04fH38dnZwNRpPff/+cnl18e3hQTLcu6cCK/OIF6oyJNUpQjYDm1eckJxvOcPIyxHl3sOW3u52tSX/yRrASPNhu9wo3xhQv03ILuokRZg3kExu01FRsrmksr9Kjyauv9/efbhfHp+eXN3fXd1/mF1fzsw8n78+vbxazo+M4L9d5RfWxHXcoQSZqHFKMSFAcioFNxXIDPJ4e3NzeTQ9m/eG4W+6M9mfFaDo9eJ0NSj/rAzeTrSDK+iQnEZusqhghJ9ksMCuUsFShnyyRz1aop8vkCsnSisWZSQNaNNAYSatLumjHapClg1FNhAQDHEZrU6xW49QqJVZIgWrIADq6lci6b3gZCjLopXrU0cMOtHxeszjkBvkQl7sE3+oZ0chPCj/Kcb4T4zJKB2FaBGlp+11JdWTkQjOUXMzrLgdNQbMFM5aDXLGjf1oHgny+m/hkAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/87706/ElasticNetRegression_Thumbnail.png","srcSet":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/0dee1/ElasticNetRegression_Thumbnail.png 750w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/8beaa/ElasticNetRegression_Thumbnail.png 1080w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/87706/ElasticNetRegression_Thumbnail.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/a66aa/ElasticNetRegression_Thumbnail.webp 750w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/65dd5/ElasticNetRegression_Thumbnail.webp 1080w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/71d4d/ElasticNetRegression_Thumbnail.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Lari Giba","bio":"Hi! I'm Lari and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"aspectRatio":0.7434944237918215,"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAbABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAMEAgH/xAAWAQEBAQAAAAAAAAAAAAAAAAABAgP/2gAMAwEAAhADEAAAAeDIop5sDKn0aCSgzv8A/8QAHhABAAICAQUAAAAAAAAAAAAAAQACAxEEEyEiMjP/2gAIAQEAAQUC3ZAE6ZGW8HtMw1M2O+tTlfI9Z//EABURAQEAAAAAAAAAAAAAAAAAABAR/9oACAEDAQE/AYU//8QAFxEBAAMAAAAAAAAAAAAAAAAAABAREv/aAAgBAgEBPwFbMf/EABwQAAIBBQEAAAAAAAAAAAAAAAABMQIQESFxkf/aAAgBAQAGPwLfpJvBhuRVIgXRNwpsxct//8QAHRABAAMAAgMBAAAAAAAAAAAAAQARIUFRMWGBcf/aAAgBAQABPyG78vaUcDUeYJrJrCUcHAhzK6pYRVinqYAdKT7Y0/Yi+8UGE//aAAwDAQACAAMAAAAQsO1C/8QAGBEBAQEBAQAAAAAAAAAAAAAAAREAECH/2gAIAQMBAT8QPF4uTBS7/8QAGBEBAQEBAQAAAAAAAAAAAAAAAQARIVH/2gAIAQIBAT8QO2vYEgOZf//EAB8QAQEAAgIBBQAAAAAAAAAAAAERACExQVFhcZGxwf/aAAgBAQABPxDlRMDhDreGljrJ8uK5DiW/WJE4rXSx7wgAOy8sEKuzsxERAF59XxirS0k0dz03g4T3iY5jH9MRu2FvejEgEAAM/9k=","sizes":"(max-width: 800px) 100vw, 800px","src":"/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg","srcSet":"/static/9eab840ebd4087faf31c48702c92af36/fd013/lari.jpg 200w,\n/static/9eab840ebd4087faf31c48702c92af36/25252/lari.jpg 400w,\n/static/9eab840ebd4087faf31c48702c92af36/2f1b1/lari.jpg 800w,\n/static/9eab840ebd4087faf31c48702c92af36/0ff54/lari.jpg 1200w,\n/static/9eab840ebd4087faf31c48702c92af36/75b83/lari.jpg 1520w"}}]}}]},"fields":{"layout":"post","slug":"/machine_learning_models/elastic_net_regression/"}},"primaryTag":"Machine Learning Models"}},"staticQueryHashes":["13077718","1316742457","1394499604","2907728042","3762220167"]}