{"componentChunkName":"component---src-templates-post-tsx","path":"/dataset_optimization/standardization/","result":{"data":{"logo":{"childImageSharp":{"gatsbyImageData":{"layout":"fixed","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/e43c6530b7b2b056de6bfee951c9e393/b181d/compasslogo.png","srcSet":"/static/e43c6530b7b2b056de6bfee951c9e393/b181d/compasslogo.png 1920w","sizes":"1920px"},"sources":[{"srcSet":"/static/e43c6530b7b2b056de6bfee951c9e393/e02c9/compasslogo.webp 1920w","type":"image/webp","sizes":"1920px"}]},"width":1920,"height":472}}},"mdx":{"body":"const _excluded = [\"components\"];\n\nfunction _extends() { _extends = Object.assign || function (target) { for (var i = 1; i < arguments.length; i++) { var source = arguments[i]; for (var key in source) { if (Object.prototype.hasOwnProperty.call(source, key)) { target[key] = source[key]; } } } return target; }; return _extends.apply(this, arguments); }\n\nfunction _objectWithoutProperties(source, excluded) { if (source == null) return {}; var target = _objectWithoutPropertiesLoose(source, excluded); var key, i; if (Object.getOwnPropertySymbols) { var sourceSymbolKeys = Object.getOwnPropertySymbols(source); for (i = 0; i < sourceSymbolKeys.length; i++) { key = sourceSymbolKeys[i]; if (excluded.indexOf(key) >= 0) continue; if (!Object.prototype.propertyIsEnumerable.call(source, key)) continue; target[key] = source[key]; } } return target; }\n\nfunction _objectWithoutPropertiesLoose(source, excluded) { if (source == null) return {}; var target = {}; var sourceKeys = Object.keys(source); var key, i; for (i = 0; i < sourceKeys.length; i++) { key = sourceKeys[i]; if (excluded.indexOf(key) >= 0) continue; target[key] = source[key]; } return target; }\n\n/* @jsxRuntime classic */\n\n/* @jsx mdx */\nconst _frontmatter = {\n \"layout\": \"post\",\n \"title\": \"When, Why, And How You Should Standardize Your Data\",\n \"author\": [\"Boris Giba\"],\n \"tags\": [\"Dataset Optimization\"],\n \"image\": \"img/Standardization/Standardization_Thumbnail.png\",\n \"date\": \"2021-07-06T11:47:10.172Z\",\n \"dateModified\": \"2021-07-06T11:47:10.172Z\",\n \"changelog\": [],\n \"draft\": false,\n \"imgCaption\": \"Photo by Dominik Schröder\",\n \"imgSource\": \"https://unsplash.com/photos/FIKD9t5_5zQ\",\n \"timeToRead\": 10,\n \"uniquePostId\": 12,\n \"excerpt\": \"Standardization is one of the most useful transformations you can apply to your dataset. What is even more important is that many models, especially regularized ones, require the data to be standardized in order to function properly. In this article, you will learn everything you need to know about standardization. You will learn why it works, when you should use it, and how you can do so with just a few lines of code.\",\n \"dependencies\": [],\n \"proficiencyLevel\": \"Beginner\"\n};\n\nconst makeShortcode = name => function MDXDefaultShortcode(props) {\n console.warn(\"Component \" + name + \" was not imported, exported, or provided by MDXProvider as global scope\");\n return mdx(\"div\", props);\n};\n\nconst ConditionalLazyPlot = makeShortcode(\"ConditionalLazyPlot\");\nconst ToolTip = makeShortcode(\"ToolTip\");\nconst HelpfulTip = makeShortcode(\"HelpfulTip\");\nconst ConditionalLazyPlotInteractive = makeShortcode(\"ConditionalLazyPlotInteractive\");\nconst InfoTip = makeShortcode(\"InfoTip\");\nconst Collapsible = makeShortcode(\"Collapsible\");\nconst layoutProps = {\n _frontmatter\n};\nconst MDXLayout = \"wrapper\";\nreturn function MDXContent(_ref) {\n let {\n components\n } = _ref,\n props = _objectWithoutProperties(_ref, _excluded);\n\n return mdx(MDXLayout, _extends({}, layoutProps, props, {\n components: components,\n mdxType: \"MDXLayout\"\n }), mdx(\"h2\", {\n \"id\": \"outline\"\n }, `Outline`), mdx(\"p\", null, `Standardization is a process from statistics where you take a dataset (or a distribution)\nand transform it such that it is centered around zero and has a standard deviation of one.\nWhat exactly this means, what problems standardization solves,\nand when you should standardize your data is exactly what you’ll learn by reading\nthis article! A lot of models (especially ones that contain regularization) require the input data\nto be standardized in order to work properly. There are lots of questions on the internet\nabout how you should correctly standardize your dataset.\nShould you use the entire dataset for\nstandardization? How should you standardize new data points?\nWhat happens if I don’t standardize my data? We will answer these questions, as well\nas many more, in this article!`), mdx(\"h2\", {\n \"id\": \"prerequisites\"\n }, `Prerequisites`), mdx(\"p\", null, `You should be comfortable with at least one machine learning model,\nlike `, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/machine_learning_models/linear_regression\"\n }, `linear regression`), ` before you read this article.\nSince standardization is especially important for regularized models,\nit also helps if you know at least one regularized model, like `, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/machine_learning_models/ridge_regression\"\n }, `ridge regression`), `.\nYou should also be familiar with the notion of training and testing datasets.\nIf you need a refresher on that, check out the article `, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/dataset_optimization/split_data_the_right_way/\"\n }, `How to Split Your Dataset the Right Way`), `.`), mdx(\"p\", null, `Additionally, we will explore a practical example from the article\n`, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/machine_learning_models/ridge_regression\"\n }, `Ridge Regression Explained, Step by Step`), ` to motivate\nthe idea of standardization. So if you’d like to have a little bit more context about the setting\nfrom this article, you can consider\nreading or skimming the article about ridge regression.\nWith that being said, let’s begin!`), mdx(\"h2\", {\n \"id\": \"the-setting\"\n }, `The Setting`), mdx(\"p\", null, `So what’s the issue? What problem does standardization solve? What can happen if we `, mdx(\"em\", {\n parentName: \"p\"\n }, `don’t`), ` standardize\nour data? Let’s look at a concrete example to see where things might go wrong. If you’ve already read\nthe article about ridge regression, then this example will feel familiar.`), mdx(\"p\", null, `Let’s say we have a dataset of figure prices where each entry in the dataset contains the age of the figure in years\nas well as its price for that age in € (or any other currency).\nWe then want to predict the price of a figure given its age using linear regression (OLS regression), to see how much the figures depreciate over time.\nThe dataset looks like this: `), mdx(ConditionalLazyPlot, _extends({}, olsVsRidgeData, {\n mdxType: \"ConditionalLazyPlot\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"1106px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"63%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAABS0lEQVQ4y52Tf3OCMAyGPUEFJy2gKIityG+9nXfb9/9u7y6ZRep2N90fOULTPEmaZDKLjnBDjVXaQuY95L6H2HeYxwWfk/0ZMXcnpEyFwjKpGLhKGwRZawHp+5eYe5PF+oQgbeCnPdyogCMVy7OZ/ciQgCJrsUx7LNYlZpF+GWYBqbQw7+HIg2V0X8xyAIb5GUHajgwKXlLD33UvQQfg+vjOwPvDGmD7PyCPS9ZCqCsWm2rUuW8YPYk8ftwdKQg5hxretkGgrlZgbkp0OGMWF+z8GHnO31ujQs3Nowoem2c1RWQdH9jjoq2LFHAa5AiLT4Y6IrdmdJhDU563OSFWFzZQ1t6m5Atvu5r/SZdZx7q/rXjUKAF6/+W2hmMyHGdBzkY3m0JrSJvjJxULnZHNT0oGUmDvFnBYPQM1JY/LoBmN1IWhY7sj9a8lfwFWWzy/hBs7RgAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/040ab5967b5b4f588dc8db80d21003fb/601b1/figurePricesData.webp 500w\", \"/static/040ab5967b5b4f588dc8db80d21003fb/80c60/figurePricesData.webp 1000w\", \"/static/040ab5967b5b4f588dc8db80d21003fb/e6a9b/figurePricesData.webp 1106w\"],\n \"sizes\": \"(max-width: 1106px) 100vw, 1106px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/040ab5967b5b4f588dc8db80d21003fb/0eb09/figurePricesData.png 500w\", \"/static/040ab5967b5b4f588dc8db80d21003fb/1263b/figurePricesData.png 1000w\", \"/static/040ab5967b5b4f588dc8db80d21003fb/22593/figurePricesData.png 1106w\"],\n \"sizes\": \"(max-width: 1106px) 100vw, 1106px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/040ab5967b5b4f588dc8db80d21003fb/22593/figurePricesData.png\",\n \"alt\": \"Figure prices dataset\",\n \"title\": \"Figure prices dataset\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `We now split our dataset into a training and testing portion\nand create a linear regression model as well as a ridge regression model:`), mdx(ConditionalLazyPlot, _extends({}, olsVsRidgeDataTrainTestOlsRidgeYears, {\n mdxType: \"ConditionalLazyPlot\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"977px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"64.2%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB6ElEQVQ4y21TaZOaQBClRLlE5uDSdS4BxXU12a2kUpVP+f+/6qVmhBXc/fDomaavN93trZjGkipYGZcNoqJBXFh5GNBgavOMZ703VRb6ilxfwdUbmHgF3b8iV28zRyunGHU+kU569hNwgzA/YMUNFkS4n1OsJlU8V2TvYdEh3Z4dQ886kP0ZXF9B9h+IeXs35hoB0wi4/pbqaLPMJDbNBd2ff6jND3iLTCCtj9hsjwioRMyOiOkJAW3gO1rW2QzyjoDNk0XcoL39Rd38hBeXLTa7HknVIeUCfCvAKwVz/I2q7rDODUImEHGJiGv49IFFprAkGgkXqNQFhb7Bs3RzdQWRZyS6c05xrhCXxgWr6haqeUexO4DVArQUYJUEKwWKpsFadvAzhQWRsGw9WxkTF5B9j7A2w0M/YGlH+Qkx7xGTDkR3SMUBIZVItwZxZWaNck1Jtyf3hn4mvzQhcAkkVkwhIAa5+QUibq4qnyis6Lz7LmC260FeelfyYxz0bO7s2af3WYtoi8Q2j7Vf5vFe4dDlpGodXOeGDbHbkpSNM15XHULbpNy4c8Ra52vvnxWOkW112UvvDGwSu35WbxNku5N7CjsN94AHbHYn+EQ4GQ22LuD0QT83Y0KV7s9uJbm8zPSj3bhyI+X/K7FA2SCQegYAAAAASUVORK5CYII=')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/341d26fabb159917cfdf433094117d5c/601b1/olsVsRidgePlotYears.webp 500w\", \"/static/341d26fabb159917cfdf433094117d5c/0ba0e/olsVsRidgePlotYears.webp 977w\"],\n \"sizes\": \"(max-width: 977px) 100vw, 977px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/341d26fabb159917cfdf433094117d5c/0eb09/olsVsRidgePlotYears.png 500w\", \"/static/341d26fabb159917cfdf433094117d5c/8cd91/olsVsRidgePlotYears.png 977w\"],\n \"sizes\": \"(max-width: 977px) 100vw, 977px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/341d26fabb159917cfdf433094117d5c/8cd91/olsVsRidgePlotYears.png\",\n \"alt\": \"OLS vs Ridge for predicting figure prices\",\n \"title\": \"OLS vs Ridge for predicting figure prices\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `Now everything looks alright, doesn’t it? So what’s the problem?`), mdx(\"h2\", {\n \"id\": \"when-to-use-standardization-and-why\"\n }, `When to Use Standardization and Why`), mdx(\"p\", null, `Before we take a look at `, mdx(\"em\", {\n parentName: \"p\"\n }, `what`), ` standardization is,\nlet’s look at what problem it solves first.\nRight now our `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X`), ` (our features) tells us the age of every figure in years, right?\nNow what happens if we change that to\n`, mdx(\"em\", {\n parentName: \"p\"\n }, `days`), `, instead? In other words, what happens if we do this:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `X_days = X * 365 # let's ignore leap years for now\n`)), mdx(\"p\", null, `and train our two models on `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_days`), ` instead of `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X`), `?\nOur dataset will still `, mdx(\"em\", {\n parentName: \"p\"\n }, `look`), ` the same and it will still hold the same information as before,\nso one would assume that our models will also look the same, right?\nBefore you take a look at the plot below, take a second to think about what we should expect.\nShould both models remain the same? Should both change?`, mdx(\"br\", {\n parentName: \"p\"\n }), `\n`, `Let’s run our two models again and see what happens:`), mdx(ConditionalLazyPlot, _extends({}, olsVsRidgeDataTrainTestOlsRidgeDays, {\n mdxType: \"ConditionalLazyPlot\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"1086px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"65.60000000000001%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAABxklEQVQ4y31Ty5KbMBAkxmaxzUPISOCABALj9WK2kqoccsz/f1WnZmwSPxIfpkYSTKunp+WtsgZzRMWAteqxnbM+YJ13WAqL2/9u4/GbNx/6qYU0IzKKekRanSCqE+e5iPJj0Lmf1lgkNXxh4dFhuHPYqB5veY8vcQ0/NXfxP3Z8SWoQVWdIO2Gre3hUkOyPEPUIZT+QaYNANvBFwzl4ARbIFn5iIE6faKafLJlHVOPyyKBEPVENVGmRFu7SjmheMxQNhHY4//jFknk0ANKJBuDH5srMoXAT8tIiyi0XvQIOMoO4HLAtDvBoALIeEVcDom7AMr0IvUgttyy0QdWNiLRj0EVssKKBpBaRGxCWjgc66+2RLS5TfUegnm8nkHDXQpYNpKqRnz4Q6I5BgrzFSt5b56Lh/sg6ksAk9L+85mctwsygPk5QVYdQEqvmzlIMSDSjcmDQR5vM6+DKgny2SAw20kLqGkLfgl0BabPRB/YhDUi1n6wRgRT9N4R5x6ZN9u/ImzMXBzuHov+OSLVsqyUTsI8vxeBt57BW3Z9XQM+Pgi6jLkhv/9raRvc8/b8tX9beraBzW/Oe2ku+ErPp6QmyHOK55d/Hv0AkAur11AAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/6add25ee2ac404f4b24991569f6170de/601b1/olsVsRidgePlotDays.webp 500w\", \"/static/6add25ee2ac404f4b24991569f6170de/80c60/olsVsRidgePlotDays.webp 1000w\", \"/static/6add25ee2ac404f4b24991569f6170de/537ed/olsVsRidgePlotDays.webp 1086w\"],\n \"sizes\": \"(max-width: 1086px) 100vw, 1086px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/6add25ee2ac404f4b24991569f6170de/0eb09/olsVsRidgePlotDays.png 500w\", \"/static/6add25ee2ac404f4b24991569f6170de/1263b/olsVsRidgePlotDays.png 1000w\", \"/static/6add25ee2ac404f4b24991569f6170de/a1954/olsVsRidgePlotDays.png 1086w\"],\n \"sizes\": \"(max-width: 1086px) 100vw, 1086px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/6add25ee2ac404f4b24991569f6170de/a1954/olsVsRidgePlotDays.png\",\n \"alt\": \"Models trained on Figure Prices Dataset in days\",\n \"title\": \"Models trained on Figure Prices Dataset in days\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `That’s interesting.. our ridge regression model now produced the same linear function as our OLS regression model!\nBut.. why? With `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_days`), `, we’ve put our `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X`), ` onto a larger scale than before. Maybe, by also putting\nour `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X`), ` onto a lower scale and looking at the outcome, we can gain a deeper insight.\nSo let’s do that and create new features `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_decades`), ` that describe the age of every figure not in years, but in decades.\nI.e. we do this:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `X_decades = X / 10\n`)), mdx(\"p\", null, `Take another second and think about how our models should look like when we train them on `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_decades`), `.\nReady? We train our models once more and we get:`), mdx(ConditionalLazyPlot, _extends({}, olsVsRidgeDataTrainTestOlsRidgeDecades, {\n mdxType: \"ConditionalLazyPlot\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"1087px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"64.4%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB8ElEQVQ4y3VTyXabQBAkEkKAgFlYJKNh2JHk2M8vl8TH/P9PVV63BMZxcihmoaemuqfa2akarrTwVI2wGBAWPSPIOwR5j8NxRJB1HEOxX/DXvkOfrah4kdUv0NUzQ5yvkOcb8uYVSXlZCGn8DNqrmIPWDn3uagbs0g6bxPDPrbA8zut/qnso9IsRSXmFn7VwKFiUFwhzgyjfEKgeO13DIygaGx7psLcimueusJDXN1x//YYsL3A2SYXoNOFQ9KC5r0YEcoInO2ykxZZR/xdEGuoa/cs7l8gJ855rdigGBMogziuIooUZfkJlPUTeIMoMoqxCklvED9D8kFrsZI1AGxzbV07bUfQI5hnxeYRsOySZZdKkoEM10mJA2fyAPLa8H2lzJ04r6K5HaCgzi2+xwUZUcMgqJDUuR+yKBlvxkQ6lTPB0j0BN2IsRh+YCN62xERYu1TdtPlnHoVeU5obkacI2qfgxuOh6XXzL2MsWp8s74uIKT9afvDnPHbJH/DQhOg5skbXf1r6b/cqXihYhPZ7qF//NsWyb6DRyd1DqpHSvm8XMtCYH0AFtv7ONSEB8mhDoHsrcsE/bD4UzMxFTIBG7jzYky5BH6bCfdXwJ7ZEjyB2uumdHAhbCdR04pVV6ftou7Ugt6D+ULGWQ9ZeU/wAOpEF4nW89oAAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/601b1/olsVsRidgePlotDecades.webp 500w\", \"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/80c60/olsVsRidgePlotDecades.webp 1000w\", \"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/77a69/olsVsRidgePlotDecades.webp 1087w\"],\n \"sizes\": \"(max-width: 1087px) 100vw, 1087px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/0eb09/olsVsRidgePlotDecades.png 500w\", \"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/1263b/olsVsRidgePlotDecades.png 1000w\", \"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/faa7d/olsVsRidgePlotDecades.png 1087w\"],\n \"sizes\": \"(max-width: 1087px) 100vw, 1087px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/bfae1c58b9d86ab8e02b48b27f1d23a4/faa7d/olsVsRidgePlotDecades.png\",\n \"alt\": \"Models trained on Figure Prices Dataset in decades\",\n \"title\": \"Models trained on Figure Prices Dataset in decades\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `Interesting! Now our ridge regression model seems to have moved in the opposite direction,\nit is almost completely flat. Our OLS regression however remained unchanged.\nWe can get some structure into these observations by plotting these three experiments side by side.\nLet’s also keep track of the model parameters (at the bottom of each plot).\nHere’s how that looks like (click on the image to zoom in!):`), mdx(\"div\", {\n className: \"largeImg\"\n }, mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"1161px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"30.4%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAYAAADDl76dAAAACXBIWXMAAAsTAAALEwEAmpwYAAABYUlEQVQY0yWMaY+aUABFiUNVCsIDlO2xw2gBBa3TJU2j6Mx0+f8/6DTSDzcnOTf3KlF7wUoOWOkRJzvhffpGPtyIuhve9icr2SOyEyI9EXY/yI93wu7OuvrOSu5xixec8kw8XEj2I4pbnNFEyNJJUYwI3SsRsiFor4j4wMLKmIuE2SpC9yucrCdoRvTNMwuRo1pyih1tWedHlM3uK6qdsvIrDK/kyUpY+AVhN+KUA/omw/JqTL9GcyRmvEX2r4h0h7HOEcEOY1OguRKnOKC41QszM0IVGZqTYsmKxabAby6YSc/MlDyJFG1dY4TPfPRqwu6G5m+ZmfG0W7o1S7/GiFoUt/yCKhLmboFq52jrAtMrCdsrVjKgipS5k/PBzpjbJYbfTId6sPvvHp2Ts7BLLNmjBM2V7PiL/PSb8vx3YtK/EbQjYXsjHd6nPPyDUXfHb0bk/j7tHq74/If0+E58eOUfAtqimO8eO+gAAAAASUVORK5CYII=')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/3987e15e382dd9fbf655e355a4d7d5cc/601b1/scalingEffectsOnRidgeAndOLS2.webp 500w\", \"/static/3987e15e382dd9fbf655e355a4d7d5cc/80c60/scalingEffectsOnRidgeAndOLS2.webp 1000w\", \"/static/3987e15e382dd9fbf655e355a4d7d5cc/79cbf/scalingEffectsOnRidgeAndOLS2.webp 1161w\"],\n \"sizes\": \"(max-width: 1161px) 100vw, 1161px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/3987e15e382dd9fbf655e355a4d7d5cc/0eb09/scalingEffectsOnRidgeAndOLS2.png 500w\", \"/static/3987e15e382dd9fbf655e355a4d7d5cc/1263b/scalingEffectsOnRidgeAndOLS2.png 1000w\", \"/static/3987e15e382dd9fbf655e355a4d7d5cc/8cf9a/scalingEffectsOnRidgeAndOLS2.png 1161w\"],\n \"sizes\": \"(max-width: 1161px) 100vw, 1161px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/3987e15e382dd9fbf655e355a4d7d5cc/8cf9a/scalingEffectsOnRidgeAndOLS2.png\",\n \"alt\": \"Effects of Scaling on Ridge Regression\",\n \"title\": \"Effects of Scaling on Ridge Regression\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `Ok, much better!\nBelow every plot, you can see the model parameters in the format `, `[intercept, slope]`, `.\nThere are a couple of things to note here:`), mdx(\"div\", null, mdx(\"ol\", null, mdx(\"li\", {\n parentName: \"ol\"\n }, `It seems that only our ridge model is sensitive to the scaling of the input data.`), mdx(\"li\", {\n parentName: \"ol\"\n }, `The ridge penalty becomes weaker when our data points are closer\ntogether, and stronger when they are further apart.`, mdx(ToolTip, {\n width: \"20em\",\n mdxType: \"ToolTip\"\n }, `For example, the points in `, mdx(\"inlineCode\", {\n parentName: \"li\"\n }, `X_days`), ` are\nfurther apart from each other than the points in `, mdx(\"inlineCode\", {\n parentName: \"li\"\n }, `X_years`), `. This is because two points\nthat were apart one year are now apart 365 days. To us humans this is the same,\nbut to the machine these are different numbers.`)), mdx(\"li\", {\n parentName: \"ol\"\n }, `The slope of the OLS model increases when our points are closer together.`), mdx(\"li\", {\n parentName: \"ol\"\n }, `The intercept term of the OLS model doesn’t change. Only the slope changes.`))), mdx(\"p\", null, `Let’s try to make sense of these points one by one.\nPoint one makes sense when we recall the loss of our two models (`, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"math math-inline\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-mathml\"\n }, mdx(\"math\", {\n parentName: \"span\",\n \"xmlns\": \"http://www.w3.org/1998/Math/MathML\"\n }, mdx(\"semantics\", {\n parentName: \"math\"\n }, mdx(\"mrow\", {\n parentName: \"semantics\"\n }, mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"bold-italic\"\n }, `θ`)), mdx(\"annotation\", {\n parentName: \"semantics\",\n \"encoding\": \"application/x-tex\"\n }, `\\\\boldsymbol{\\\\theta}`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-html\",\n \"aria-hidden\": \"true\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"0.69444em\",\n \"verticalAlign\": \"0em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord boldsymbol\",\n \"style\": {\n \"marginRight\": \"0.03194em\"\n }\n }, `θ`))))))), ` are our model parameters):`), mdx(\"div\", {\n \"className\": \"math math-display\"\n }, mdx(\"span\", {\n parentName: \"div\",\n \"className\": \"katex-display\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-mathml\"\n }, mdx(\"math\", {\n parentName: \"span\",\n \"xmlns\": \"http://www.w3.org/1998/Math/MathML\",\n \"display\": \"block\"\n }, mdx(\"semantics\", {\n parentName: \"math\"\n }, mdx(\"mtable\", {\n parentName: \"semantics\",\n \"rowspacing\": \"0.24999999999999992em\",\n \"columnalign\": \"right left\",\n \"columnspacing\": \"0em\"\n }, mdx(\"mtr\", {\n parentName: \"mtable\"\n }, mdx(\"mtd\", {\n parentName: \"mtr\"\n }, mdx(\"mstyle\", {\n parentName: \"mtd\",\n \"scriptlevel\": \"0\",\n \"displaystyle\": \"true\"\n }, mdx(\"mrow\", {\n parentName: \"mstyle\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `O`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `L`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `S`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `L`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `o`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `s`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `s`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `(`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `y`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"separator\": \"true\"\n }, `,`), mdx(\"msub\", {\n parentName: \"mrow\"\n }, mdx(\"mi\", {\n parentName: \"msub\"\n }, `y`), mdx(\"mrow\", {\n parentName: \"msub\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `p`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `r`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `e`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `d`))), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `)`)))), mdx(\"mtd\", {\n parentName: \"mtr\"\n }, mdx(\"mstyle\", {\n parentName: \"mtd\",\n \"scriptlevel\": \"0\",\n \"displaystyle\": \"true\"\n }, mdx(\"mrow\", {\n parentName: \"mstyle\"\n }, mdx(\"mrow\", {\n parentName: \"mrow\"\n }), mdx(\"mo\", {\n parentName: \"mrow\"\n }, `=`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `M`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `S`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `E`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `(`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `y`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"separator\": \"true\"\n }, `,`), mdx(\"msub\", {\n parentName: \"mrow\"\n }, mdx(\"mi\", {\n parentName: \"msub\"\n }, `y`), mdx(\"mrow\", {\n parentName: \"msub\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `p`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `r`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `e`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `d`))), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `)`))))), mdx(\"mtr\", {\n parentName: \"mtable\"\n }, mdx(\"mtd\", {\n parentName: \"mtr\"\n }, mdx(\"mstyle\", {\n parentName: \"mtd\",\n \"scriptlevel\": \"0\",\n \"displaystyle\": \"true\"\n }, mdx(\"mrow\", {\n parentName: \"mstyle\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `R`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `i`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `d`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `g`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `e`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `L`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `o`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `s`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `s`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `(`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `y`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"separator\": \"true\"\n }, `,`), mdx(\"msub\", {\n parentName: \"mrow\"\n }, mdx(\"mi\", {\n parentName: \"msub\"\n }, `y`), mdx(\"mrow\", {\n parentName: \"msub\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `p`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `r`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `e`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `d`))), mdx(\"mo\", {\n parentName: \"mrow\",\n \"separator\": \"true\"\n }, `,`), mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"bold-italic\"\n }, `θ`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `)`)))), mdx(\"mtd\", {\n parentName: \"mtr\"\n }, mdx(\"mstyle\", {\n parentName: \"mtd\",\n \"scriptlevel\": \"0\",\n \"displaystyle\": \"true\"\n }, mdx(\"mrow\", {\n parentName: \"mstyle\"\n }, mdx(\"mrow\", {\n parentName: \"mrow\"\n }), mdx(\"mo\", {\n parentName: \"mrow\"\n }, `=`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `M`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `S`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `E`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `(`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `y`), mdx(\"mo\", {\n parentName: \"mrow\",\n \"separator\": \"true\"\n }, `,`), mdx(\"msub\", {\n parentName: \"mrow\"\n }, mdx(\"mi\", {\n parentName: \"msub\"\n }, `y`), mdx(\"mrow\", {\n parentName: \"msub\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `p`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `r`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `e`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `d`))), mdx(\"mo\", {\n parentName: \"mrow\",\n \"stretchy\": \"false\"\n }, `)`), mdx(\"mo\", {\n parentName: \"mrow\"\n }, `+`), mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"normal\"\n }, `∣`), mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"normal\"\n }, `∣`), mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"bold-italic\"\n }, `θ`), mdx(\"mi\", {\n parentName: \"mrow\",\n \"mathvariant\": \"normal\"\n }, `∣`), mdx(\"msubsup\", {\n parentName: \"mrow\"\n }, mdx(\"mi\", {\n parentName: \"msubsup\",\n \"mathvariant\": \"normal\"\n }, `∣`), mdx(\"mn\", {\n parentName: \"msubsup\"\n }, `2`), mdx(\"mn\", {\n parentName: \"msubsup\"\n }, `2`))))))), mdx(\"annotation\", {\n parentName: \"semantics\",\n \"encoding\": \"application/x-tex\"\n }, `\n\\\\begin{aligned}\nOLS Loss(y, y_{pred}) &= MSE(y, y_{pred}) \\\\\\\\\nRidge Loss(y, y_{pred}, \\\\boldsymbol{\\\\theta}) &= MSE(y, y_{pred}) + ||\\\\boldsymbol{\\\\theta}||_2^2\n\\\\end{aligned}\n\n`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-html\",\n \"aria-hidden\": \"true\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"3.024108em\",\n \"verticalAlign\": \"-1.262054em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mtable\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"col-align-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"1.762054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-3.922054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.02778em\"\n }\n }, `O`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `L`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.05764em\"\n }\n }, `S`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `L`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `o`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `s`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `s`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mopen\"\n }, `(`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mpunct\"\n }, `,`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.16666666666666666em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"msupsub\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.3361079999999999em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.5500000000000003em\",\n \"marginLeft\": \"-0.03588em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `p`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\",\n \"style\": {\n \"marginRight\": \"0.02778em\"\n }\n }, `r`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `e`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `d`))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.286108em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n })))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mclose\"\n }, `)`))), mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.397946em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.00773em\"\n }\n }, `R`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `i`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `d`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `g`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `e`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `L`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `o`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `s`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `s`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mopen\"\n }, `(`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mpunct\"\n }, `,`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.16666666666666666em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"msupsub\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.3361079999999999em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.5500000000000003em\",\n \"marginLeft\": \"-0.03588em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `p`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\",\n \"style\": {\n \"marginRight\": \"0.02778em\"\n }\n }, `r`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `e`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `d`))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.286108em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n })))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mpunct\"\n }, `,`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.16666666666666666em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord boldsymbol\",\n \"style\": {\n \"marginRight\": \"0.03194em\"\n }\n }, `θ`))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mclose\"\n }, `)`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"1.262054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n }))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"col-align-l\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"1.762054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-3.922054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mrel\"\n }, `=`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.10903em\"\n }\n }, `M`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.05764em\"\n }\n }, `S`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.05764em\"\n }\n }, `E`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mopen\"\n }, `(`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mpunct\"\n }, `,`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.16666666666666666em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"msupsub\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.3361079999999999em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.5500000000000003em\",\n \"marginLeft\": \"-0.03588em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `p`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\",\n \"style\": {\n \"marginRight\": \"0.02778em\"\n }\n }, `r`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `e`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `d`))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.286108em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n })))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mclose\"\n }, `)`))), mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.397946em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mrel\"\n }, `=`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.10903em\"\n }\n }, `M`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.05764em\"\n }\n }, `S`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.05764em\"\n }\n }, `E`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mopen\"\n }, `(`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mpunct\"\n }, `,`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.16666666666666666em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `y`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"msupsub\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.3361079999999999em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.5500000000000003em\",\n \"marginLeft\": \"-0.03588em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `p`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\",\n \"style\": {\n \"marginRight\": \"0.02778em\"\n }\n }, `r`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `e`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal mtight\"\n }, `d`))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.286108em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n })))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mclose\"\n }, `)`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2222222222222222em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mbin\"\n }, `+`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2222222222222222em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, `∣`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, `∣`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord boldsymbol\",\n \"style\": {\n \"marginRight\": \"0.03194em\"\n }\n }, `θ`))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, `∣`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, `∣`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"msupsub\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.8641079999999999em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.4530000000000003em\",\n \"marginLeft\": \"0em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, `2`))), mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-3.113em\",\n \"marginRight\": \"0.05em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"2.7em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"sizing reset-size6 size3 mtight\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mtight\"\n }, `2`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.247em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n }))))))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"1.262054em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n })))))))))))), mdx(\"p\", null, `The loss of our OLS model is just the MSE, which does not care about the size of our model\nparameters. The ridge loss, however, contains an L2-penalty, which punishes large model weights.\nNow let’s take a look at the slope of our OLS regression model once more.\nIt is -116, which is pretty large (in the absolutes) in the plot on the right, where we use `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_decades`), `.\nThis makes sense because our data points are very close to each other (absolutely speaking). Each data point\nis maybe 0.02 units away from the next, so our slope has to be relatively large.\nBut the ridge penalty `, mdx(\"em\", {\n parentName: \"p\"\n }, `discourages`), ` large model parameters! This is why our ridge model\nbecomes so flat in the right example. `), mdx(\"p\", null, `Similarly, in the left example, where we use `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_days`), `,\nthe data points are pretty far apart from each other, which means that our slope\nshould be particularly small (in the absolutes), which is true since it is just -0.03 for OLS regression.\nAnd the ridge penalty `, mdx(\"em\", {\n parentName: \"p\"\n }, `encourages`), ` small model parameters,\nso it does not see a problem with our (now) small model parameters, so it does not regularize them further!`), mdx(\"p\", null, `This issue appears very frequently in machine learning and it can be a real pain to deal with\nbecause you might not suspect that the scaling of your data is causing these issues.\nThis issue also affects almost all regularized models, because they all share some sort of\nregularization term that punishes large model weights.`), mdx(\"p\", null, `Here is when standardization comes into play. Standardization allows us to put all of our features\non the same scale, resolving this issue. `, mdx(\"mark\", null, `So `, mdx(\"em\", {\n parentName: \"p\"\n }, `when`), ` and `, mdx(\"em\", {\n parentName: \"p\"\n }, `why`), ` should we use standardization?`)), mdx(HelpfulTip, {\n mdxType: \"HelpfulTip\"\n }, mdx(\"p\", null, `Generally speaking, standardization should be used when your model has a regularization term\nor is otherwise sensitive to the scaling of the input features. Standardization transforms\nall features onto the same scaling, thereby ensuring that regularization and other\nscaling-sensitive operations work properly. `)), mdx(\"p\", null, `Now that we know `, mdx(\"em\", {\n parentName: \"p\"\n }, `when`), ` we should use standardization and `, mdx(\"em\", {\n parentName: \"p\"\n }, `why`), `,\nlet’s take a look at `, mdx(\"em\", {\n parentName: \"p\"\n }, `how`), ` exactly it works.`), mdx(\"h2\", {\n \"id\": \"how-standardization-works\"\n }, `How Standardization Works`), mdx(\"p\", null, `What standardization does is it puts every feature in a dataset on the same scale.\nSo how does it work?\nIt involves the following two steps:`), mdx(\"div\", null, mdx(\"ol\", null, mdx(\"li\", {\n parentName: \"ol\"\n }, `subtract the mean (the `, mdx(\"em\", {\n parentName: \"li\"\n }, `expected value`), `)`, mdx(ToolTip, {\n width: \"30em\",\n mdxType: \"ToolTip\"\n }, `Generally the mean is not the same as the expected value,\nespecially if you’re dealing with probability distributions and random variables.\nThe expected value for a discrete random variable is the sum of every `, mdx(\"em\", {\n parentName: \"li\"\n }, `event outcome`), ` times\nthe `, mdx(\"em\", {\n parentName: \"li\"\n }, `probability of that event occurring`), `. In our dataset we’re not dealing with\nany probabilities, so we can just drop that part from our equation. The event outcome\nin our case is just the y-value of a data point, i.e. the price of a figure for some age. That’s why\nin this case the mean and the expected value are the same.`), ` of the dataset from every feature point`), mdx(\"li\", {\n parentName: \"ol\"\n }, `divide every feature point by the standard deviation (`, mdx(\"em\", {\n parentName: \"li\"\n }, `std`), ` for short) of the dataset`))), mdx(\"p\", null, `And that’s it! In essence, you can implement standardization in just one line of code:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `X_standardized = (X - np.mean(X) ) / np.std(X)\n`)), mdx(\"p\", null, `Often times you will see it being written like that:`), mdx(\"div\", {\n \"className\": \"math math-display\"\n }, mdx(\"span\", {\n parentName: \"div\",\n \"className\": \"katex-display\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-mathml\"\n }, mdx(\"math\", {\n parentName: \"span\",\n \"xmlns\": \"http://www.w3.org/1998/Math/MathML\",\n \"display\": \"block\"\n }, mdx(\"semantics\", {\n parentName: \"math\"\n }, mdx(\"mrow\", {\n parentName: \"semantics\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `z`), mdx(\"mo\", {\n parentName: \"mrow\"\n }, `=`), mdx(\"mfrac\", {\n parentName: \"mrow\"\n }, mdx(\"mrow\", {\n parentName: \"mfrac\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `x`), mdx(\"mo\", {\n parentName: \"mrow\"\n }, `−`), mdx(\"mi\", {\n parentName: \"mrow\"\n }, `μ`)), mdx(\"mi\", {\n parentName: \"mfrac\"\n }, `σ`))), mdx(\"annotation\", {\n parentName: \"semantics\",\n \"encoding\": \"application/x-tex\"\n }, `z = \\\\frac{x - \\\\mu}{\\\\sigma}`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-html\",\n \"aria-hidden\": \"true\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"0.43056em\",\n \"verticalAlign\": \"0em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.04398em\"\n }\n }, `z`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mrel\"\n }, `=`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2777777777777778em\"\n }\n })), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"1.9463300000000001em\",\n \"verticalAlign\": \"-0.686em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mopen nulldelimiter\"\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mfrac\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-t vlist-t2\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"1.2603300000000002em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-2.314em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `σ`))), mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-3.23em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"frac-line\",\n \"style\": {\n \"borderBottomWidth\": \"0.04em\"\n }\n })), mdx(\"span\", {\n parentName: \"span\",\n \"style\": {\n \"top\": \"-3.677em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"pstrut\",\n \"style\": {\n \"height\": \"3em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `x`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2222222222222222em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mbin\"\n }, `−`), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mspace\",\n \"style\": {\n \"marginRight\": \"0.2222222222222222em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `μ`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-s\"\n }, `​`)), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist-r\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"vlist\",\n \"style\": {\n \"height\": \"0.686em\"\n }\n }, mdx(\"span\", {\n parentName: \"span\"\n }))))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mclose nulldelimiter\"\n }))))))), mdx(\"p\", null, `Here, `, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"math math-inline\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-mathml\"\n }, mdx(\"math\", {\n parentName: \"span\",\n \"xmlns\": \"http://www.w3.org/1998/Math/MathML\"\n }, mdx(\"semantics\", {\n parentName: \"math\"\n }, mdx(\"mrow\", {\n parentName: \"semantics\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `μ`)), mdx(\"annotation\", {\n parentName: \"semantics\",\n \"encoding\": \"application/x-tex\"\n }, `\\\\mu`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-html\",\n \"aria-hidden\": \"true\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"0.625em\",\n \"verticalAlign\": \"-0.19444em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\"\n }, `μ`))))), ` is the `, mdx(\"em\", {\n parentName: \"p\"\n }, `expected value`), ` of our dataset (which in this case is just the mean)\nand `, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"math math-inline\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-mathml\"\n }, mdx(\"math\", {\n parentName: \"span\",\n \"xmlns\": \"http://www.w3.org/1998/Math/MathML\"\n }, mdx(\"semantics\", {\n parentName: \"math\"\n }, mdx(\"mrow\", {\n parentName: \"semantics\"\n }, mdx(\"mi\", {\n parentName: \"mrow\"\n }, `σ`)), mdx(\"annotation\", {\n parentName: \"semantics\",\n \"encoding\": \"application/x-tex\"\n }, `\\\\sigma`)))), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"katex-html\",\n \"aria-hidden\": \"true\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"base\"\n }, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"strut\",\n \"style\": {\n \"height\": \"0.43056em\",\n \"verticalAlign\": \"0em\"\n }\n }), mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"mord mathnormal\",\n \"style\": {\n \"marginRight\": \"0.03588em\"\n }\n }, `σ`))))), ` is the standard deviation of our dataset.`), mdx(\"p\", null, `Nice!\nLet’s take a look at what happens\nwhen we standardize our `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_days`), `, our `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_years`), `, and our `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_decades`), `.\nHere’s an image showing the three datasets on the left and their standardized variants on the right (For this, I visualized\nthe data points as bars because the plot is quite large, and bars are easier to see than points):`), mdx(\"div\", {\n className: \"largeImg\"\n }, mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"858px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"83.2%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAARCAYAAADdRIy+AAAACXBIWXMAAAsTAAALEwEAmpwYAAAD7klEQVQ4y02U629UVRTFJ8x0OqUz087rzr33nDOvdjqvTmvttIXSdwEBH8GAKCgICJa2M8OUFkMixhC/GBOjRv3QGLSkSrG08lAkIeGDJsb4V/3MPVOUDyc72WdnZZ219jquSGaIULpCOF3R1S6M4fTCmWHCXSOEUoOYhUmiXSO6Z46+hRg4hCqNo4pjhGUvnaJEKNGPkRvD1RpJ4+lQeDoTeDoV7g6Jp0PgjWVptcu0xHoQ5VnajW5ajByyuol59luix28QOtSgc+IioZkqoYPLRDIVXK3RjAZrCSV3TkoDe40cPtmP1y5j9x1oAsYLyIUtZG0T1dhEXb5D4uoT1JXH2JfuEOkabjJ0/8fQOUk8HVIzaxV9tFglDbg75jDMIxa3EIu3kYvriEvfI6vbiIUtzPM/EM4M4WqLdeMNp/BG0ro6DFvCKXxmkTY1gE/0IfoP4I9n8ZpFVHUbWd1AVpuAqv6L7tnvrRHtHsElemewilOI0rSuib79xLuGEfkxzMKE1s/uncUuTjXnKkdJjZ6ge+IU2al3SOw7hdzzJnL4GHbvDK6A6KXdKuC3i7THc4QcsGMfY8zOEUgO6jurMEGn6scv+7BeuYoxepJoaYZwpoLfyuO3ivhlP7HuPbicZ7qDUuvn9pt4xQuolSckP/wLUbuPVw1i9eylPd6DJ5ZDNh4haj9jza9hXlgl/vbXmGdWib/7XVNDX7Sr6W44rdfFp15E1u6RvPYHiZUn+FIjmD178Jv5psvzm8jFn5DVH5ELt1BLv6LqD7Au3Gq67DDcFRS4OxPs8pu0ygHk0mOS1/9GXX2KNzGEmR1ht5HF4+xk7QGivomsbyAW15GXH+qeeXG9ybBDlQnYJRwt/WaOYHqYxLlV0vMbpC6uEcyO6/SEUwNa08Tpb1BnvkKd/RJ1+nM9mzy7ijz1BUZuHy4nVo6bZn4cuzRF18hr2L3TxHNjuhfPOf0ZrJ05q7yfdP8sqdIE6fIkorAPIzOEkalgl6afaZjCExS0JQZJLT8k+tISnva43k0nNVZhXJviRE+8v4Z5/AbR2TnCoycJlA4S6HuVQOWEdr2ZZScdAQtfokLi2p/Ejn2Ke3dMG+VsgF2c2AHMI6v3kJe3UVe2ELXbyPp9ZP3Bc9FzshxK4gnaTcAPnhJ7/RPc7TuAHQrrOUAxv4mYX0csrCHmbiJr28iFu5jnbv6f5Wc76ACmPvoH443PcLdFtBTuoGgCGln9ZNX4DdXYRi3d1U6r5d9RjUfYcxv623M5+XOQI+lBYvlJkkevY0+eJ5wcILLzH8ryLEZ2L9HsKOrwMurIEupIA3m4hnp5BXVkBXmwjlWc5F9vVyLErz66VgAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/a918ed910d0df37eed7278dbbebffd7b/601b1/notStandardVsStandard.webp 500w\", \"/static/a918ed910d0df37eed7278dbbebffd7b/72a6f/notStandardVsStandard.webp 858w\"],\n \"sizes\": \"(max-width: 858px) 100vw, 858px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/a918ed910d0df37eed7278dbbebffd7b/0eb09/notStandardVsStandard.png 500w\", \"/static/a918ed910d0df37eed7278dbbebffd7b/fd891/notStandardVsStandard.png 858w\"],\n \"sizes\": \"(max-width: 858px) 100vw, 858px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/a918ed910d0df37eed7278dbbebffd7b/fd891/notStandardVsStandard.png\",\n \"alt\": \"non-standardized vs standardized data\",\n \"title\": \"non-standardized vs standardized data\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `You can see how the three standardized datasets are exactly the same!\nWe can visualize this process to better understand\nthe two steps individually (we’ll only include the training data for now):`), mdx(ConditionalLazyPlotInteractive, _extends({}, olsVsRidgeStandardize, {\n mdxType: \"ConditionalLazyPlotInteractive\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"887px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"67.80000000000001%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB9klEQVQ4y4WT6W7aQBSFHfa1zOIV7zZUhFI1yg8TO0BR1PYdAiokEm3f/wlONRMMNqjpj9G9Hs98c84cW6kRD2IMhhNwbwbqTsG8T6DOFNSeysrdGYh9izr15VpRL/t8KHnTM8Ygzq0Ei8394RgtI0DXitGzR2ibIarEPYFymKjVgYvqcU5pqZGEVT44qIgXAxdK3wQjdxixH4jMJ4T8CSP2HZR9RoU4BYUummqMvj1DR4/fFDZ5KB/yExs0QMeZINa/Yake8Bi+IBvusdQPGPIH3JChXJOra2oR2PgO9vgebS26BorRtMYItDVSdY+5v0FibZBqO5h8XgLWqYc68dHjH+V1ScsC2NVHBWAAZWDBV1fI+B5zb4vE2iLT9rCugH4poBPw0nKFOvDZEgvvN9J4h8TcvAssurtS+Aa04bMVHt1feIh/IjG3yPSXI9D+J/CUctcoW64Q+2zZfUYavWIR/oFJE9xQ+32Fl8BGAZhqIpSttCxDYYm83wb7j+WOVkiZ+NJywNdYagdkwQ6ptcNCfYVrrNDxpqgPvKswSpaJPUGD+mjxUH5LNepC1+8RqmtEzhqh8VX2XP+Cph6jrUYQQoRSsb+kUH6c/GxBLJKnMk8mWqWOVCzCEAeJPym3XCP+sT+rVIqyi/KF9VMt9iWb+Z7z/F9rKWml34W+HwAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/29d4d0feb26dd607b3ad648d47fa09bb/601b1/standardizationVisTrain.webp 500w\", \"/static/29d4d0feb26dd607b3ad648d47fa09bb/6f880/standardizationVisTrain.webp 887w\"],\n \"sizes\": \"(max-width: 887px) 100vw, 887px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/29d4d0feb26dd607b3ad648d47fa09bb/0eb09/standardizationVisTrain.png 500w\", \"/static/29d4d0feb26dd607b3ad648d47fa09bb/713f0/standardizationVisTrain.png 887w\"],\n \"sizes\": \"(max-width: 887px) 100vw, 887px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/29d4d0feb26dd607b3ad648d47fa09bb/713f0/standardizationVisTrain.png\",\n \"alt\": \"Standardization visualized\",\n \"title\": \"Standardization visualized\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `In the plot above, you can press the button at the top to perform the standardization steps.`), mdx(\"p\", null, `If we now run OLS regression and ridge regression on our standardized dataset (any of the three,\nsince they are now the same), we get:`), mdx(ConditionalLazyPlot, _extends({}, olsVsRidgeDataTrainTestOlsRidgeStandardized, {\n mdxType: \"ConditionalLazyPlot\"\n }), mdx(\"p\", null, mdx(\"span\", {\n parentName: \"p\",\n \"className\": \"gatsby-resp-image-wrapper\",\n \"style\": {\n \"position\": \"relative\",\n \"display\": \"block\",\n \"marginLeft\": \"auto\",\n \"marginRight\": \"auto\",\n \"maxWidth\": \"897px\"\n }\n }, `\n `, mdx(\"span\", {\n parentName: \"span\",\n \"className\": \"gatsby-resp-image-background-image\",\n \"style\": {\n \"paddingBottom\": \"70.4%\",\n \"position\": \"relative\",\n \"bottom\": \"0\",\n \"left\": \"0\",\n \"backgroundImage\": \"url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB+UlEQVQ4y31T2ZKbMBCkbIzNYluIG9tICGzY9fqobKpSeUjl//+qUzNgjn3Iw5QkGPX09KitldSwfQVa3biGG1d469d1aPpvNd6SGpuoGnI5pvs+rClgVj0R6Rvk6YpQ37A/tIjLO+LygbR6Qp4+sBRdLt2xX6tfYCkK3lvTirYcfy45cYzXhSmBlezWTXSBl57hBCUsShTHdwT6Dll8MQO7L7AOSo7vbY0taziRQfL8her2m0E7wMM79tkFSWqQZw08abCg1ggw/A+g1MwqrO+4PP6w5qzhLm/hJWduaxsVUOUNp9MV+8iwDAsxF9+enB2pECQa7c+/TMwKik9Q7PIGq0Bzy5usxS49I8kMVPkBERvWa+l3IG5ewQmmTNWgqxWqT/jHK4RusT1fsBLdYBY0BGqn/IE4OeOQ1ZCJwjpQ8OqG5RgY+3oYlkUaBMUVIm+5ZWeSSAMhJpZ3RNA+oK5fyGKDMFFwwwK2LHswNYBai33B7dKEZk/IVwxOQtuigJu12KQtHGEg4wZpViPOe2B/fEI8ZWZ46BjOJ6hHragYF+y0dPwaImywDQ2/2cEpxIQmTHajy7Tv2qbvNZ9Ha1ZsP6cv5qUXrOXcjvxsXs5g3UIztEx7KkSrw3p2wQA+OcTMJjwwnOr23V5ReWcfkyxT+60GW+pZ/j+dQ1pxls9aRgAAAABJRU5ErkJggg==')\",\n \"backgroundSize\": \"cover\",\n \"display\": \"block\"\n }\n }), `\n `, mdx(\"picture\", {\n parentName: \"span\"\n }, `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/e1268d0567a7a002018a4cd88c734fdf/601b1/olsVsRidgePlotStandard.webp 500w\", \"/static/e1268d0567a7a002018a4cd88c734fdf/b5541/olsVsRidgePlotStandard.webp 897w\"],\n \"sizes\": \"(max-width: 897px) 100vw, 897px\",\n \"type\": \"image/webp\"\n }), `\n `, mdx(\"source\", {\n parentName: \"picture\",\n \"srcSet\": [\"/static/e1268d0567a7a002018a4cd88c734fdf/0eb09/olsVsRidgePlotStandard.png 500w\", \"/static/e1268d0567a7a002018a4cd88c734fdf/d3e70/olsVsRidgePlotStandard.png 897w\"],\n \"sizes\": \"(max-width: 897px) 100vw, 897px\",\n \"type\": \"image/png\"\n }), `\n `, mdx(\"img\", {\n parentName: \"picture\",\n \"className\": \"gatsby-resp-image-image\",\n \"src\": \"/static/e1268d0567a7a002018a4cd88c734fdf/d3e70/olsVsRidgePlotStandard.png\",\n \"alt\": \"Models trained on Figure Prices Dataset standardized\",\n \"title\": \"Models trained on Figure Prices Dataset standardized\",\n \"loading\": \"lazy\",\n \"decoding\": \"async\",\n \"style\": {\n \"width\": \"100%\",\n \"height\": \"100%\",\n \"margin\": \"0\",\n \"verticalAlign\": \"middle\",\n \"position\": \"absolute\",\n \"top\": \"0\",\n \"left\": \"0\"\n }\n }), `\n `), `\n `))), mdx(\"p\", null, `Nice! This looks quite a bit less regularized than our initial ridge model, but it just goes to show\nhow much of a change standardization can make! Now let’s look at how we can actually implement it.`), mdx(\"h3\", {\n \"id\": \"standardizing-multiple-features\"\n }, `Standardizing Multiple Features`), mdx(\"p\", null, `In our example, we only have one feature (the age of a figure).\nBut standardization also works if you have multiple features!\nLet’s say we were given not only the age of every figure but also the number of people\nwho have preordered that figure. Our dataset would look like this:`), mdx(\"table\", null, mdx(\"thead\", {\n parentName: \"table\"\n }, mdx(\"tr\", {\n parentName: \"thead\"\n }, mdx(\"th\", {\n parentName: \"tr\",\n \"align\": null\n }, `Sample Index`), mdx(\"th\", {\n parentName: \"tr\",\n \"align\": null\n }, `Age (years)`), mdx(\"th\", {\n parentName: \"tr\",\n \"align\": null\n }, `#Preorders`), mdx(\"th\", {\n parentName: \"tr\",\n \"align\": null\n }, `Price (€)`))), mdx(\"tbody\", {\n parentName: \"table\"\n }, mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1.25`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1000`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `40`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `2`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `300`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `42`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `3`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `0.75`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1200`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `46`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `4`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1.5`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `500`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `37`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `5`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1.75`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `600`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `40`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `6`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `1.5`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `250`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `38`)), mdx(\"tr\", {\n parentName: \"tbody\"\n }, mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `7`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `0.75`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `900`), mdx(\"td\", {\n parentName: \"tr\",\n \"align\": null\n }, `39.8`)))), mdx(\"p\", null, `Now we want to standardize this dataset.\nTo do this we can just standardize every feature separately,\nmeaning we standardize the age-column and then we standardize\nthe #preorders-column. This is how we can apply standardization\nwhen we have more than one feature. If you are using something like scikit-learn’s `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `,\nthis will be done automatically.`), mdx(\"h2\", {\n \"id\": \"how-to-implement-standardization\"\n }, `How to Implement Standardization`), mdx(\"p\", null, `I want to present you with three ways to implement standardization.\nWe can implement it 1. by hand, 2. by using scikit-learn’s `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `,\nand 3. by utilizing scikit-learn’s `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `Pipeline`), `. Let’s start with the first way.`), mdx(\"h3\", {\n \"id\": \"implementing-standardization-manually\"\n }, `Implementing Standardization Manually`), mdx(\"p\", null, `So let’s say we have our features `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X`), `. How can we standardize them?\nHere’s the process:`), mdx(\"ol\", null, mdx(\"li\", {\n parentName: \"ol\"\n }, `Split the dataset into train and test subsets (using something like scikit-learn’s `, mdx(\"inlineCode\", {\n parentName: \"li\"\n }, `train_test_split`), `-function)`), mdx(\"li\", {\n parentName: \"ol\"\n }, `Compute mean and std of the train set`), mdx(\"li\", {\n parentName: \"ol\"\n }, `Standardize train `, mdx(\"em\", {\n parentName: \"li\"\n }, `and`), ` test set using the values computed in step 2`)), mdx(\"p\", null, `In code, it would look like this:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `X_train, X_test, y_train, y_test = train_test_split(\n X, y, test_size=0.33, random_state=42)\n\nX_train_mean = np.mean(X_train)\nX_train_std = np.std(X_train)\n\nX_train_standardized = (X_train - X_train_mean) / X_train_std\nX_test_standardized = (X_test - X_train_mean) / X_train_std\n`)), mdx(\"p\", null, `Then we can train and use our model as follows:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `ridge = Ridge()\nridge.fit(X_train_standardized, y_train)\nprint(ridge.intercept_, ridge.coef_[0])\n# outputs: 40.7 -1.56\n\nridge_predictions = ridge.predict(X_test_standardized)\nprint(ridge_predictions)\n# outputs: [40.48 41.35 39.61]\n`)), mdx(InfoTip, {\n mdxType: \"InfoTip\"\n }, mdx(\"p\", null, `Two things are important here.`), mdx(\"ol\", null, mdx(\"li\", {\n parentName: \"ol\"\n }, `Always only use the training set to compute the mean and std,\nand standardize the test set using these exact values.`), mdx(\"li\", {\n parentName: \"ol\"\n }, `Always standardize data that the model should predict using\nthe mean and std of the training set.`)), mdx(Collapsible, {\n title: \"Explanation\",\n mdxType: \"Collapsible\"\n }, mdx(\"p\", null, `The first point is important because if we were to use testing data to compute\nthe mean and std, we would have extracted some information about our testing data\ninto the mean and std. This is sometimes referred to as `, mdx(\"em\", {\n parentName: \"p\"\n }, `information leakage`), `.\nBasically, small amounts of information from our testing set `, mdx(\"em\", {\n parentName: \"p\"\n }, `leak`), ` into the mean\nand the std, changing it slightly. Since we then standardize using this information,\nour model might receive some minor information about our testing set during training,\nwhich should not happen.\nNow, in the vast majority of cases, this `, mdx(\"em\", {\n parentName: \"p\"\n }, `information leakage`), ` will be tiny and negligible.\nBut we should still not use testing data for computing the mean and std, because\nit creates a good habit and in the rare case where this might actually matter,\nwe will be on the safe side.\nThere are very few cases in general where the testing data is touched at all before the final\nmodel evaluation (one example is scaling the data to a fixed interval, also known as `, mdx(\"em\", {\n parentName: \"p\"\n }, `min-max-scaling`), ` or `, mdx(\"em\", {\n parentName: \"p\"\n }, `normalization`), `).`), mdx(\"p\", null, `The second point is important because the standardization that is applied to the data should not change\nafter the initial training since\nour model will have adapted to this particular data scaling. If different values for the mean\nand the std are used every time the model is fed with data, then the scaling will vary each and every time,\nmaking the predictions of our model inconsistent.\nI like to think about this like a chef who is well-known for their seasoned omelette.\nNow if that chef were to use a different seasoning for every omelette, then the dishes would still\ntaste somewhat similar, but the results would nevertheless be different.\nSo if you want your chef (your model) to always make consistently seasoned omelettes (consistent predictions),\nthen make sure to give them the exact same seasonings (same mean and std) every time.`))), mdx(\"p\", null, `Note that when we standardize our test data using the mean and standard deviation\nof the training data, the resulting dataset will usually not have a mean of exactly\n0 and not have an std of exactly 1. That is ok. The important part is that the transformation\nstays consistent across training and testing sets.`), mdx(\"p\", null, `Alternatively, instead of computing the mean and standard deviation ourselves,\nwe can use scikit-learn’s `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `.`), mdx(\"h3\", {\n \"id\": \"implementing-standardization-using-scikit-learns-standardscaler\"\n }, `Implementing Standardization using Scikit-Learn’s `, mdx(\"inlineCode\", {\n parentName: \"h3\"\n }, `StandardScaler`)), mdx(\"p\", null, `We can create a `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `-object and train it using only our training dataset.\nAll this “training” does is compute the mean and std of the data we feed it and store those values\nfor later use.\nThen we can use our scaler to standardize our testing set.\nIt looks like this in code:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `from sklearn.preprocessing import StandardScaler\n\nscaler = StandardScaler()\nscaler.fit(X_train)\nX_train_standardized = scaler.transform(X_train)\nX_test_standardized = scaler.transform(X_test)\n`)), mdx(\"p\", null, `Instead of writing `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `scaler.fit(X_train)`), ` and `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_train_standardized = scaler.transform(X_train)`), ` we can also just write\n`, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `X_train_standardized = scaler.fit_transform(X_train)`), ` to save us one line of code.`), mdx(\"p\", null, `Now we can again train our model:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `ridge = Ridge()\nridge.fit(X_train_standardized, y_train)\nprint(ridge.intercept_, ridge.coef_[0])\n# outputs: 40.7 -1.56\n\nridge_predictions = ridge.predict(X_test_standardized)\nprint(ridge_predictions)\n# outputs: [40.48 41.35 39.61]\n`)), mdx(\"p\", null, `The results are exactly the same.`), mdx(\"h3\", {\n \"id\": \"implementing-standardization-using-scikit-learns-pipeline\"\n }, `Implementing Standardization using Scikit-Learn’s `, mdx(\"inlineCode\", {\n parentName: \"h3\"\n }, `Pipeline`)), mdx(\"p\", null, `In the previous two examples, we had to be extra careful to always use only\nthe training data to compute our standardization parameters (the mean and the std)\nand also always standardize new data that goes through our model.\nBecause this is a bit cumbersome and very prone to errors, there is an easier solution: `, mdx(\"em\", {\n parentName: \"p\"\n }, `pipelines`), `.\nPipelines allow us to perform these computations automatically, by sequentially chaining\ntransformations, as well as one regular machine learning model, together.\nThey look like this:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `from sklearn.pipeline import Pipeline\npipeline = Pipeline([('scaler', StandardScaler()),\n ('ridge', Ridge())])\n`)), mdx(\"p\", null, `Every item inside of a pipeline has to be a tuple containing a name as well as a transformation\n(meaning it needs to implement a `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `.transform`), `-method), apart from the very last item, which has to be a tuple containing a name and a model.\nWe can then access our items like this:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `pipeline[\"scaler\"]\n# or\npipeline[0]\n`)), mdx(\"p\", null, `When we call `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `.fit`), ` on our pipeline, what will happen is that we will first fit our\n`, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), ` and then we will transform our data using said `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `.\nThen we will use that transformed data to fit our ridge regression model. When we call `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `.predict`), `\non our pipeline, we will first transform our data using our scaler, and then use that transformed data\nto make our predictions with our ridge model. `), mdx(\"p\", null, `When fitting or predicting, all of the transformations are applied `, mdx(\"em\", {\n parentName: \"p\"\n }, `on the fly`), `, saving us\nthe hassle of applying the transformations manually. Pipelines take care of the rules\nmentioned above, saving us some lines of code and reducing the possibility of errors in our code.\nThere is also `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `make_pipeline`), `, which makes creating pipelines even easier:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `from sklearn.pipeline import make_pipeline\npipeline = Pipeline([StandardScaler(), Ridge()])\n`)), mdx(\"p\", null, `Here, we don’t even have to specify the names of our items, because `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `make_pipeline`), ` will\nuse the lowercase types of each class as the names automatically (f.e “ridge” for `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `Ridge`), `).\nRegardless of whether we use `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `Pipeline`), ` or `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `make_pipeline`), `, the training process will be\nexactly the same:`), mdx(\"pre\", null, mdx(\"code\", {\n parentName: \"pre\",\n \"className\": \"language-python\"\n }, `pipeline = make_pipeline(StandardScaler(), Ridge()) \npipeline.fit(X_train, y_train)\nprint(pipeline[1].intercept_, pipeline[1].coef_[0])\n# 40.7 -1.56\npipeline_predictions = ridge.predict(X_test_standardized)\nprint(pipeline_predictions)\n# [40.48 41.35 39.61]\n`)), mdx(HelpfulTip, {\n mdxType: \"HelpfulTip\"\n }, mdx(\"p\", null, `As you can see, the results of these three methods are exactly the same! `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `make_pipeline`), ` saves us a few lines\nof code and reduces the potential for errors, because it it always uses the same `, mdx(\"inlineCode\", {\n parentName: \"p\"\n }, `StandardScaler`), `-object\nto transform our data, and it also makes sure to use the same\ndata to train our scaler and our model, which is something we would have to take care of manually if we were\nto not use a pipeline.\nBecause of these reasons, I would recommend you use pipelines\nin practical projects instead of doing the processing manually.`)), mdx(\"h2\", {\n \"id\": \"futher-reading\"\n }, `Futher Reading`), mdx(\"p\", null, `Good job on finishing this article!\nStandardization (and data preprocessing in general) can be easily overlooked,\nbut is extremely important if you want your models to make reliable predictions.`), mdx(\"p\", null, `Apart from standardization, there is also another popular data transformation\nknown as `, mdx(\"em\", {\n parentName: \"p\"\n }, `normalization`), ` or `, mdx(\"em\", {\n parentName: \"p\"\n }, `min-max scaling`), `.\nWith normalization the values are transformed into a fixed interval of a minimum\nand a maximum value, hence the name. If you’re interested, you can read more about\nnormalization `, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/dataset_optimization/normalization\"\n }, `here`), `.`), mdx(\"p\", null, `Learning about standardization is nice, but using it in practice is even better.\nIf you want to dive deeper into some practical use cases, check out the articles about\n`, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/machine_learning_models/ridge_regression\"\n }, `ridge regression`), `\nand `, mdx(\"a\", {\n parentName: \"p\",\n \"href\": \"/machine_learning_models/lasso_regression\"\n }, `lasso regression`), `.`));\n}\n;\nMDXContent.isMDXComponent = true;","excerpt":"Standardization is one of the most useful transformations you can apply to your dataset. What is even more important is that many models, especially regularized ones, require the data to be standardized in order to function properly. In this article, you will learn everything you need to know about standardization. You will learn why it works, when you should use it, and how you can do so with just a few lines of code.","tableOfContents":{"items":[{"url":"#outline","title":"Outline"},{"url":"#prerequisites","title":"Prerequisites"},{"url":"#the-setting","title":"The Setting"},{"url":"#when-to-use-standardization-and-why","title":"When to Use Standardization and Why"},{"url":"#how-standardization-works","title":"How Standardization Works","items":[{"url":"#standardizing-multiple-features","title":"Standardizing Multiple Features"}]},{"url":"#how-to-implement-standardization","title":"How to Implement Standardization","items":[{"url":"#implementing-standardization-manually","title":"Implementing Standardization Manually"},{"url":"#implementing-standardization-using-scikit-learns-standardscaler","title":"Implementing Standardization using Scikit-Learn's StandardScaler"},{"url":"#implementing-standardization-using-scikit-learns-pipeline","title":"Implementing Standardization using Scikit-Learn's Pipeline"}]},{"url":"#futher-reading","title":"Futher Reading"}]},"frontmatter":{"title":"When, Why, And How You Should Standardize Your Data","userDate":"6 July 2021","date":"2021-07-06T11:47:10.172Z","dateModified":"2021-07-06T11:47:10.172Z","changelog":[],"tags":["Dataset Optimization"],"timeToRead":10,"excerpt":"Standardization is one of the most useful transformations you can apply to your dataset. What is even more important is that many models, especially regularized ones, require the data to be standardized in order to function properly. In this article, you will learn everything you need to know about standardization. You will learn why it works, when you should use it, and how you can do so with just a few lines of code.","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACZUlEQVQozx2PyW7TUAAA/Q90yZ4uNGspVFSKRDnQbI6z2Vn8vNux4+X5PTtJGzWhSwht0kNLRQ8gVISExKk3vhGVyxzmNEPQKm6qqCa7lAhJwS4LTom3i5xdAFaBNfPAKnJWibdJEVYlVFdwQ8ENFdOa3zaGBK2ihoJqkkuJDiXCMm+VOKsAzCJn5VnziDXzrEkKsCJASnKrkluTcV32ahJq6QMCmEPvZGJAH/RgW4W8NeT6vmL7tGx1dQ8YnuF4VFcBxoAzsNJ3geawhq+643bPJw4LjfOLi+XNzej4eHZ5eXZ+uVguZ7PZ/Ze78enp9Oz87vYWITydTpfLxWA4+nx1dTqZXF8vNPeYCERTucMiyQgKHEmWz0iW0PcE0xNMTIsmZ2DQc4Huis9m0FJswRwAHTOy09F9IryRXQnuBGPp3YMPVaB3DZ9WIKNCCvTw/Ntw8VjuakzPoxWXVlBb9xkVMSpmNO85OxTPRDZ3wxvZQDQR2cwWqh3RxF3d//jwZM8f0eK39eknB6c0r7Uks6W6HQ23NcQobkvziFA8HYpn/jMd337FslyZrOeKncHXv9z4QZh81+Z/quoQYt/BowqrtzTclGFdgg0ZPT8HY6lQ7D/j6ezrXHwjHd99D+ZPpHtPj3/Ux78qygmQDaB7ddFuSLAqOhXBoSSXiCXehnZywVgmEE0FoqkXwZdr4URoc29jvxTJHCYOyvslvsLbTclqSHZNsineaiqIVlG7h4ntvaPkfj68tbcaTqxHkuuR5FokuRZOrAa2VkPbK4GtYCz55l2RBHpdtGu8VeH6JDBaKuLN4T+RFNmEkCrl1gAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/d95f1cf4daefb962c345773fa1506879/87706/Standardization_Thumbnail.png","srcSet":"/static/d95f1cf4daefb962c345773fa1506879/0dee1/Standardization_Thumbnail.png 750w,\n/static/d95f1cf4daefb962c345773fa1506879/8beaa/Standardization_Thumbnail.png 1080w,\n/static/d95f1cf4daefb962c345773fa1506879/87706/Standardization_Thumbnail.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/d95f1cf4daefb962c345773fa1506879/a66aa/Standardization_Thumbnail.webp 750w,\n/static/d95f1cf4daefb962c345773fa1506879/65dd5/Standardization_Thumbnail.webp 1080w,\n/static/d95f1cf4daefb962c345773fa1506879/71d4d/Standardization_Thumbnail.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Boris Giba","bio":"Hi! I'm Boris and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAQFBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAB25bcya1Bcz84HRg//8QAGxAAAwACAwAAAAAAAAAAAAAAAQIDABESMTP/2gAIAQEAAQUCo3FFo4OEbG1R8u5SXZh4f//EABURAQEAAAAAAAAAAAAAAAAAAAEg/9oACAEDAQE/ASP/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAeEAABBAEFAAAAAAAAAAAAAAABAAIQETESIUFRcf/aAAgBAQAGPwIkZTQ/Sb6ilvxBIzDPF//EABoQAQACAwEAAAAAAAAAAAAAAAERIQAQMUH/2gAIAQEAAT8hQ1PjHRkxa9E644V8JdD3BEk1JiRlbnUf/9oADAMBAAIAAwAAABADAID/xAAWEQEBAQAAAAAAAAAAAAAAAAABESD/2gAIAQMBAT8QAjcf/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAR/9oACAECAQE/EFdIT//EAB4QAQACAQQDAAAAAAAAAAAAAAERIQAQMUFRYYGx/9oACAEBAAE/EClkAFglYJ8Xl1kNwXZjrQ1xCEcPBl8E6A89+sAII7JzjXgEBSYlyYrZm+d/uVsWluf/2Q==","aspectRatio":1,"src":"/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg","srcSet":"/static/ba9e631272af1b925d6491b30df3b36f/b3d27/boris.jpg 40w,\n/static/ba9e631272af1b925d6491b30df3b36f/5a735/boris.jpg 80w,\n/static/ba9e631272af1b925d6491b30df3b36f/6e81a/boris.jpg 120w,\n/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg 800w,\n/static/ba9e631272af1b925d6491b30df3b36f/97bce/boris.jpg 1878w","sizes":"(max-width: 800px) 100vw, 800px"}}]}}],"imgCaption":"Photo by Dominik Schröder","imgSource":"https://unsplash.com/photos/FIKD9t5_5zQ","uniquePostId":12,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner"}},"relatedPosts":{"totalCount":2,"edges":[{"node":{"id":"d9274609-29df-5679-92c0-1f0d2c752677","excerpt":"Standardization is one of the most useful transformations you can apply to your dataset. What is even more important is that many models, especially regularized ones, require the data to be standardized in order to function properly. In this article, you will learn everything you need to know about standardization. You will learn why it works, when you should use it, and how you can do so with just a few lines of code.","frontmatter":{"title":"When, Why, And How You Should Standardize Your Data","timeToRead":10,"date":"2021-07-06T11:47:10.172Z"},"fields":{"slug":"/dataset_optimization/standardization/"}}},{"node":{"id":"025f12b7-e505-5eff-8617-17e9e76707b8","excerpt":"If you are just starting out in machine learning and building your first real models, you will have to split your dataset into a train set as well as a test set. But what benefits does this splitting yield? How can you split your dataset optimally? In this article, we will go through these questions and explore why splitting your dataset makes sense and how you can split your dataset properly.","frontmatter":{"title":"How to Split Your Dataset the Right Way","timeToRead":15,"date":"2021-05-01T18:09:26.968Z"},"fields":{"slug":"/dataset_optimization/split_data_the_right_way/"}}}]}},"pageContext":{"slug":"/dataset_optimization/standardization/","prev":{"excerpt":"Elastic net is a combination of the two most popular regularized variants of linear regression: ridge and lasso. Ridge utilizes an L2 penalty and lasso uses an L1 penalty. With elastic net, you don't have to choose between these two models, because elastic net uses both the L2 and the L1 penalty! In practice, you will almost always want to use elastic net over ridge or lasso, and in this article you will learn everything you need to know to do so, successfully!","frontmatter":{"title":"Elastic Net Regression Explained, Step by Step","tags":["Machine Learning Models"],"date":"2021-06-26T08:28:02.234Z","dateModified":"2021-06-26T08:28:02.234Z","changelog":[],"draft":false,"excerpt":"Elastic net is a combination of the two most popular regularized variants of linear regression: ridge and lasso. Ridge utilizes an L2 penalty and lasso uses an L1 penalty. With elastic net, you don't have to choose between these two models, because elastic net uses both the L2 and the L1 penalty! In practice, you will almost always want to use elastic net over ridge or lasso, and in this article you will learn everything you need to know to do so, successfully!","uniquePostId":11,"timeToRead":7,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACWElEQVQozxXNW0/TUAAA4MYYQ0A2oGVr19tO29Oedr3fd7p1WwtjbBMRjItBBgIGUGQ8aZCor/iq/mDj9wc+oml4optt8Eq1ztegs86Cap2jgWakpVfua0kh2rGeFLgYDcb7e7P5u/Orw7fzOC9Xaxzh4r7XGcotT7F8aHnIjZATa06EvNgIcbq9l48P28XYxgWKeyjI6gBJLR+YQZVTCA/3zBAzss7CFgcNVkHQTVQ3kU1fNr2GpDHQUdJdwelUGmCdlYHu/G98XGGaREMy9KCtOqGIrCaykrwALW+NldZZwGn2KtMkBdXIdmtqQLKA4hUeOaJmaU5UAzpB0ZKMXKA7myIMs973Hz8/LxaX1x+PT04fH38dnZwNRpPff/+cnl18e3hQTLcu6cCK/OIF6oyJNUpQjYDm1eckJxvOcPIyxHl3sOW3u52tSX/yRrASPNhu9wo3xhQv03ILuokRZg3kExu01FRsrmksr9Kjyauv9/efbhfHp+eXN3fXd1/mF1fzsw8n78+vbxazo+M4L9d5RfWxHXcoQSZqHFKMSFAcioFNxXIDPJ4e3NzeTQ9m/eG4W+6M9mfFaDo9eJ0NSj/rAzeTrSDK+iQnEZusqhghJ9ksMCuUsFShnyyRz1aop8vkCsnSisWZSQNaNNAYSatLumjHapClg1FNhAQDHEZrU6xW49QqJVZIgWrIADq6lci6b3gZCjLopXrU0cMOtHxeszjkBvkQl7sE3+oZ0chPCj/Kcb4T4zJKB2FaBGlp+11JdWTkQjOUXMzrLgdNQbMFM5aDXLGjf1oHgny+m/hkAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/87706/ElasticNetRegression_Thumbnail.png","srcSet":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/0dee1/ElasticNetRegression_Thumbnail.png 750w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/8beaa/ElasticNetRegression_Thumbnail.png 1080w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/87706/ElasticNetRegression_Thumbnail.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/a66aa/ElasticNetRegression_Thumbnail.webp 750w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/65dd5/ElasticNetRegression_Thumbnail.webp 1080w,\n/static/73e4a37e19eb2668d1fddc5ebf3bd7c1/71d4d/ElasticNetRegression_Thumbnail.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Boris Giba","bio":"Hi! I'm Boris and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"aspectRatio":1,"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAQFBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAB25bcya1Bcz84HRg//8QAGxAAAwACAwAAAAAAAAAAAAAAAQIDABESMTP/2gAIAQEAAQUCo3FFo4OEbG1R8u5SXZh4f//EABURAQEAAAAAAAAAAAAAAAAAAAEg/9oACAEDAQE/ASP/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAeEAABBAEFAAAAAAAAAAAAAAABAAIQETESIUFRcf/aAAgBAQAGPwIkZTQ/Sb6ilvxBIzDPF//EABoQAQACAwEAAAAAAAAAAAAAAAERIQAQMUH/2gAIAQEAAT8hQ1PjHRkxa9E644V8JdD3BEk1JiRlbnUf/9oADAMBAAIAAwAAABADAID/xAAWEQEBAQAAAAAAAAAAAAAAAAABESD/2gAIAQMBAT8QAjcf/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAR/9oACAECAQE/EFdIT//EAB4QAQACAQQDAAAAAAAAAAAAAAERIQAQMUFRYYGx/9oACAEBAAE/EClkAFglYJ8Xl1kNwXZjrQ1xCEcPBl8E6A89+sAII7JzjXgEBSYlyYrZm+d/uVsWluf/2Q==","sizes":"(max-width: 800px) 100vw, 800px","src":"/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg","srcSet":"/static/ba9e631272af1b925d6491b30df3b36f/fd013/boris.jpg 200w,\n/static/ba9e631272af1b925d6491b30df3b36f/25252/boris.jpg 400w,\n/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg 800w,\n/static/ba9e631272af1b925d6491b30df3b36f/0ff54/boris.jpg 1200w,\n/static/ba9e631272af1b925d6491b30df3b36f/06655/boris.jpg 1600w,\n/static/ba9e631272af1b925d6491b30df3b36f/97bce/boris.jpg 1878w"}}]}}]},"fields":{"layout":"post","slug":"/machine_learning_models/elastic_net_regression/"}},"next":{"excerpt":"Gradient descent is one of the most popular algorithms to train machine learning models. However, many of the popular machine learning models like lasso regression or support vector machines contain loss functions that are not differentiable. Because of this, regular gradient descent can not be used. One of the most commonly utilized techniques to circumvent this issue is to use subgradients instead of regular gradients. And in this article, you will learn how it's done.","frontmatter":{"title":"Subgradient Descent Explained, Step by Step","tags":["Machine Learning Math"],"date":"2021-07-27T19:49:21.503Z","dateModified":"2021-08-22T18:51:14.860Z","changelog":[["2023-01-17T19:31:25.928Z","Fixed a mistake in the explanations of coding exercises #1 and #2. Previously, I wrote that ridge regression converges faster than lasso regression. What I meant to say was that ridge regression reaches much lower loss values than lasso regression much quicker. In the exercise examples, lasso, not ridge, does indeed converge faster, but it plateaus at a much higher loss value, while ridge continues to decrease in loss. Also fixed a small typo in the solution for exercise 2. Big thanks to Андрей for pointing these mistakes out!"],["2021-07-31T14:28:41.400Z","Added interactive, runnable code blocks."],["2021-08-21T17:17:59.435Z","Reworked section 'The Fatal Flaw' to more correctly describe the issue of weights not being set exactly to zero. Added quiz questions and programming exercises."],["2021-08-22T18:51:14.860Z","Added Jupyter notebook."]],"draft":false,"excerpt":"Gradient descent is one of the most popular algorithms to train machine learning models. However, many of the popular machine learning models like lasso regression or support vector machines contain loss functions that are not differentiable. Because of this, regular gradient descent can not be used. One of the most commonly utilized techniques to circumvent this issue is to use subgradients instead of regular gradients. And in this article, you will learn how it's done.","uniquePostId":13,"timeToRead":18,"tutorialId":null,"tutorialEntryIdx":null,"dependencies":[],"proficiencyLevel":"Beginner","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACaElEQVQozyXMW3eSAAAAYH5C1pB5gTkRxRt3kYvgBUXUCQqK4n2bzu3UcnPHtszWOT1YPdTpJ/fQ9wM+wCvTgyrvVHiDT5kKaxVzvVph7rQmuthXibEuLuxmTxP6VenqojjRxSaf9gzF1iSdTQALnZlo9Fhj1PRZKRup0WhHoSd1eaqLtsp5VaGtMG0pY0pZS6H7GlchYxUq1srhVRIFVCLuqpmumLAFjE/AAg535GxHJqx80i4QdQblsKBI4mo2KiTCcurMEKimSDgqrVMokEqSIfC0kEQ29ZQroJ6MfbXI3Xp2eHm+X6++PO/2n55+/Py1uX+/3z3tHjeH/Ytd4nUqOm8qAE3ncZwAfWCTQb0CPpAwmwpsltNvr4dBS9tvP67n49nAmbr2zHPdtjGyza7KLprqdtoBYmGYZfLJJOl7A/lPQuvl8uZyNrbNgdno64pdKwxa1UFTmzjtWd+SEyE1Fbm2qrc9Y6HnACIWo7FovahSBPvWF5jPL7fb7fF4vL1Z/f3ze728Ph5/PD0+vO4/P9zdGBx+aWrb0cWwkFo1BAA8CQX8MIYgllYySuVyWZekkiwVW7ox71krt33nda+c1sq1Pkz7h/Xo++1gdaGMjcKowgEJNJGK4SAYBt8Fk1GUjcfwyHkAgn2+oB8MkXGsJVLDCrdsCPemdFXnHYUcltlJhZ7rPMBTrMiJERg9hRAQhAMQkjxHOTyeiqKBAAJBCIVhppB1lWxHynRl0qtwowo7q+WGKgkgwUg4GAHBMOQPQ2AYPAn/L6g4VmXSGpMmsJgfhM9CSDOfaeTwrph25Kyrkj0p/Q8tyKY2MJ2klwAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/8d667dbc302a08dd3745c90bfb78dfd9/87706/SubGradientDescent_Thumbnail2.png","srcSet":"/static/8d667dbc302a08dd3745c90bfb78dfd9/0dee1/SubGradientDescent_Thumbnail2.png 750w,\n/static/8d667dbc302a08dd3745c90bfb78dfd9/8beaa/SubGradientDescent_Thumbnail2.png 1080w,\n/static/8d667dbc302a08dd3745c90bfb78dfd9/87706/SubGradientDescent_Thumbnail2.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/8d667dbc302a08dd3745c90bfb78dfd9/a66aa/SubGradientDescent_Thumbnail2.webp 750w,\n/static/8d667dbc302a08dd3745c90bfb78dfd9/65dd5/SubGradientDescent_Thumbnail2.webp 1080w,\n/static/8d667dbc302a08dd3745c90bfb78dfd9/71d4d/SubGradientDescent_Thumbnail2.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5625}}},"author":[{"id":"Boris Giba","bio":"Hi! I'm Boris and I run this website. You can find more information in the \"About\"-tab.","avatar":{"children":[{"fluid":{"aspectRatio":1,"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAQFBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAB25bcya1Bcz84HRg//8QAGxAAAwACAwAAAAAAAAAAAAAAAQIDABESMTP/2gAIAQEAAQUCo3FFo4OEbG1R8u5SXZh4f//EABURAQEAAAAAAAAAAAAAAAAAAAEg/9oACAEDAQE/ASP/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAeEAABBAEFAAAAAAAAAAAAAAABAAIQETESIUFRcf/aAAgBAQAGPwIkZTQ/Sb6ilvxBIzDPF//EABoQAQACAwEAAAAAAAAAAAAAAAERIQAQMUH/2gAIAQEAAT8hQ1PjHRkxa9E644V8JdD3BEk1JiRlbnUf/9oADAMBAAIAAwAAABADAID/xAAWEQEBAQAAAAAAAAAAAAAAAAABESD/2gAIAQMBAT8QAjcf/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAR/9oACAECAQE/EFdIT//EAB4QAQACAQQDAAAAAAAAAAAAAAERIQAQMUFRYYGx/9oACAEBAAE/EClkAFglYJ8Xl1kNwXZjrQ1xCEcPBl8E6A89+sAII7JzjXgEBSYlyYrZm+d/uVsWluf/2Q==","sizes":"(max-width: 800px) 100vw, 800px","src":"/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg","srcSet":"/static/ba9e631272af1b925d6491b30df3b36f/fd013/boris.jpg 200w,\n/static/ba9e631272af1b925d6491b30df3b36f/25252/boris.jpg 400w,\n/static/ba9e631272af1b925d6491b30df3b36f/2f1b1/boris.jpg 800w,\n/static/ba9e631272af1b925d6491b30df3b36f/0ff54/boris.jpg 1200w,\n/static/ba9e631272af1b925d6491b30df3b36f/06655/boris.jpg 1600w,\n/static/ba9e631272af1b925d6491b30df3b36f/97bce/boris.jpg 1878w"}}]}}]},"fields":{"layout":"post","slug":"/machine_learning_math/subgradient_descent/"}},"primaryTag":"Dataset Optimization"}},"staticQueryHashes":["13077718","1316742457","1394499604","2907728042","3762220167"]}