[{"content":"Welcome to Ciência Programada! This is a space where we explore the intersection of science and programming, offering resources, tutorials and discussions on how to apply programming techniques to solve scientific problems.\nSee the latest posts below for content updates. You can also search for specific topics using the search bar in the top right corner of the page. Or, if you prefer, you can browse the posts by category, tag or series using the links in the footer or in the previous sentence.\nProject developed by Francisco Bustamante.\n","date":"2026-10-28","externalUrl":null,"permalink":"/","section":"","summary":"","title":"","type":"page"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/posts/","section":"Blog","summary":"","title":"Blog","type":"posts"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"Quando a matemática simbólica encontra seu limite, entra a numérica. Esta série percorre o SciPy e o NumPy resolvendo o que o SymPy não resolve: equações sem solução fechada, integrais que não têm primitiva, derivadas a partir de dados medidos, ajustes de curva com incerteza, sistemas de equações diferenciais e a transformada de Fourier — sempre partindo de problemas concretos de física, química e engenharia.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-10-28","externalUrl":null,"permalink":"/series/do-simbolo-ao-numero/","section":"Séries","summary":"","title":"Do Símbolo ao Número","type":"series"},{"content":"Chemist, data scientist and Python programmer.\nLearn more about me in the About section.\n","date":"2026-10-28","externalUrl":null,"permalink":"/authors/flsbustamante/","section":"Authors","summary":"","title":"Francisco Bustamante","type":"authors"},{"content":"When symbolic mathematics reaches its limit, numerical mathematics takes over. This series goes through SciPy and NumPy, solving what SymPy cannot: equations with no closed-form solution, integrals with no antiderivative, derivatives from measured data, curve fits with uncertainty, systems of differential equations and the Fourier transform, always starting from concrete problems in physics, chemistry and engineering.\nThe articles are listed below, in the order they were written.\n","date":"2026-10-28","externalUrl":null,"permalink":"/series/from-symbol-to-number/","section":"Series","summary":"","title":"From Symbol to Number","type":"series"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/tags/optimization/","section":"tags","summary":"","title":"optimization","type":"tags"},{"content":" Sizing a cylindrical tank of minimum cost, given a fixed volume, is a classic exercise: you write the cost as a function of diameter and height, use the volume constraint to eliminate one of the two variables, differentiate what is left and set it to zero. Three lines of algebra and the problem is solved. But that elegance depends on a condition that goes unnoticed: you could isolate one variable as a function of the other in a closed expression. What if you could not, because the relation between the variables is too complicated, or because there is more than one constraint, or because the cost function has no tractable symbolic derivative? The calculation that solves everything by differentiating and setting to zero, by hand, breaks down.\nWhy Does Numerical Optimization Matter? # Engineering: sizing equipment and structures almost always involves several design variables and simultaneous constraints (cost, safety, available space), and rarely fits in a single algebraic substitution. Data science and machine learning: training a model is, at heart, minimizing a loss function with thousands (or millions) of parameters, impossible to solve by differentiating by hand. Economics and operations research: allocating limited resources optimally is a constrained optimization problem almost by definition. Natural sciences: fitting the parameters of a nonlinear model to experimental data is minimizing the error between model and measurement, that is, optimization disguised as curve fitting. From the f\u0026rsquo;=0 Criterion to scipy.optimize # The SymPy optimization article solved \\(f(x) = x^3 - 2x^2 + x\\) the symbolic way: diff to find the derivative, solve to set it to zero. The result is a list with all the critical points at once, \\(x=1/3\\) (local maximum) and \\(x=1\\) (local minimum):\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 - 2 * x**2 + x critical_points = sp.solve(sp.diff(f_sym, x), x) print(critical_points) [1/3, 1] scipy.optimize.minimize_scalar attacks the same kind of problem in a different way: instead of solving a symbolic equation, it does a numerical search for a minimum inside an interval. The difference in philosophy already shows up here, because minimize_scalar only looks for a minimum, so it will never return the local maximum at \\(1/3\\), no matter the interval:\nimport sympy as sp from scipy.optimize import minimize_scalar x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 - 2 * x**2 + x f_num = sp.lambdify(x, f_sym) res = minimize_scalar(f_num, bounds=(0.5, 1.5), method=\u0026#34;bounded\u0026#34;) print(res.x, res.fun, res.success) 1.0000000114867313 0.0 True It matches \\(x=1\\), the local minimum, within the expected numerical tolerance.\nWhy method=\u0026quot;bounded\u0026quot; minimize_scalar has three methods, and the choice is not immaterial:\nMethod Needs an interval? Notes brent (default) No, accepts an optional bracket Combines golden-section search with parabolic interpolation; it is the analog of brentq for minimization golden No Pure golden-section search; slower, of historical value bounded Yes, required The only one that respects bounds and never leaves the interval The distinction matters: bounds is only honored by bounded. With brent, the optimizer may perfectly well wander outside the interval you thought you had imposed, and in a problem with physical meaning (a concentration, an absolute temperature) that usually ends in the function being evaluated at meaningless values.\nWhat if I want the maximum? scipy.optimize has no maximize function. It is not missed: maximizing \\(f\\) is the same as minimizing \\(-f\\), since flipping the function upside down swaps peaks for valleys without moving any of them. It is the most used trick in the whole library, and it solves the maximum at \\(1/3\\) that minimize_scalar \u0026ldquo;cannot find\u0026rdquo;:\nimport sympy as sp from scipy.optimize import minimize_scalar x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 - 2 * x**2 + x f_num = sp.lambdify(x, f_sym) res_max = minimize_scalar(lambda t: -f_num(t), bounds=(0, 0.8), method=\u0026#34;bounded\u0026#34;) print(\u0026#34;x of the maximum:\u0026#34;, res_max.x) print(\u0026#34;f(x) at the maximum:\u0026#34;, -res_max.fun) # undo the sign! x of the maximum: 0.3333331957133562 f(x) at the maximum: 0.1481481481481292 It matches \\(x = 1/3\\) and \\(f(1/3) = 4/27 \\approx 0.1481\\).\nTwo pitfalls in this pattern, both easy to fall into:\nres.fun comes with the sign flipped. It holds the value of \\(-f\\), which is what was minimized. Forgetting the - when reading the result gives a maximum with the wrong sign, and nothing warns you. The interval starts to matter in a different way. Here I used \\((0,\\ 0.8)\\) and not \\((0.5,\\ 1.5)\\), because the maximum is at \\(1/3\\), outside the interval of the previous example. Changing the objective usually requires changing the interval too. This same trick reappears in the next article four times in a row: there the goal is to maximize profit and range, and all the functions are written as minus_profit, minus_conversion, -projectile_range. The \u0026ldquo;minus\u0026rdquo; prefix in the name is not style: it is the trick.\nWhy \\([0.5, 1.5]\\) and not \\([0, 1]\\) Restricting the search to \\([0, 1]\\) would seem more natural (it contains both critical points), but this function has a coincidence, \\(f(0) = f(1) = 0\\), and the two ends of the interval tie at the minimum value. That does not invalidate the method, but it makes the check less clean (the optimizer converges to near one of the ends, with less precision, because the minimum is on the edge). Choosing an interval where the minimum falls strictly inside, such as \\([0.5, 1.5]\\), avoids this ambiguity.\nThe Cylindrical Tank, Without Isolating Any Variable # The same SymPy optimization article cited above, besides the introductory example, also posed a more elaborate problem: sizing a cylindrical tank of fixed volume (\\(V\\)) while minimizing the construction cost, given the cost per unit area of the side (\\(c_s\\)) and of the top/bottom (\\(c_t\\)). It is worth revisiting it in full, with the same symbols.\nLet us recall where the cost function comes from. With a bit of basic geometry (the lateral area of a cylinder is \\(\\pi D L\\), and the area of the top plus the bottom is \\(2 \\times \\frac{\\pi}{4}D^2 = \\frac{\\pi}{2}D^2\\)), we arrive at the cost function:\n$$ f = c_s \\pi D L + c_t \\frac{\\pi}{2} D^2 $$Geometry of the cylindrical tank: diameter D and height L determine the lateral area and the areas of the top and bottom. This function is subject to the volume being fixed, \\(\\frac{\\pi}{4}D^2 L = V\\), the same constraint as in the original article.\nExample 1: Revalidating the published result Statement: With \\(c_s = c_t = 1\\) and \\(V=10\\), find the diameter that minimizes the cost, isolating \\(L\\) as a function of \\(V\\) first, the same way as the original article.\nSolution:\nimport numpy as np from scipy.optimize import minimize_scalar V, cs, ct = 10.0, 1.0, 1.0 def cost_1var(D): L = 4 * V / (np.pi * D**2) return cs * np.pi * D * L + ct * (np.pi / 2) * D**2 res_1var = minimize_scalar(cost_1var, bounds=(0.01, 20), method=\u0026#34;bounded\u0026#34;) print(\u0026#34;Optimal D:\u0026#34;, res_1var.x) Optimal D: 2.335088713257265 The symbolic value published in the original article was \\(2.33508864988147\\); it matches up to the 6th decimal place, within the optimizer\u0026rsquo;s default tolerance.\nThe following plot shows the cost curve as a function of \\(D\\), with the minimum found marked:\nimport matplotlib.pyplot as plt import numpy as np from scipy.optimize import minimize_scalar V, cs, ct = 10.0, 1.0, 1.0 def cost_1var(D): L = 4 * V / (np.pi * D**2) return cs * np.pi * D * L + ct * (np.pi / 2) * D**2 res_1var = minimize_scalar(cost_1var, bounds=(0.01, 20), method=\u0026#34;bounded\u0026#34;) Ds = np.linspace(0.5, 6, 200) fig, ax = plt.subplots() ax.plot(Ds, cost_1var(Ds)) ax.plot(res_1var.x, res_1var.fun, \u0026#34;o\u0026#34;, color=\u0026#34;black\u0026#34;) ax.set_xlabel(\u0026#34;D (m)\u0026#34;) ax.set_ylabel(\u0026#34;cost\u0026#34;) The tank cost curve as a function of diameter, with the minimum found by minimize_scalar marked. This part only confirms that the numerical route agrees with the symbolic route when both are possible. The central point of the article is the next step: solving the same problem without isolating \\(L\\), as if that elimination were not convenient (or did not exist). That is an optimization in two variables, \\((D, L)\\), subject to an equality constraint:\nimport numpy as np from scipy.optimize import minimize V, cs, ct = 10.0, 1.0, 1.0 def cost_2var(v): D, L = v return cs * np.pi * D * L + ct * (np.pi / 2) * D**2 def volume_constraint(v): D, L = v return (np.pi / 4) * D**2 * L - V res_2var = minimize( cost_2var, x0=[1.0, 1.0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;eq\u0026#34;, \u0026#34;fun\u0026#34;: volume_constraint}, bounds=[(1e-6, None), (1e-6, None)], tol=1e-12, ) print(res_2var.x, res_2var.success) [2.33508865 2.33508864] True method=\u0026quot;SLSQP\u0026quot; (\u0026quot;sequential least squares programming\u0026quot;) is SciPy\u0026rsquo;s default method for constrained optimization. The constraint enters as a dictionary: 'type': 'eq' says that volume_constraint must be exactly zero at the solution, the same volume equation as in the original article, only without solving for \\(L\\) before optimizing. D and L come out practically equal, confirming the relation \\(D=L\\) that the original article had derived symbolically for equal costs.\nExample 2: Second validation: distinct costs Statement: Repeat the previous problem with the side cost equal to twice the top cost (\\(c_s=2\\), \\(c_t=1\\), \\(V=10\\) remain).\nSolution:\nimport numpy as np from scipy.optimize import minimize V, cs, ct = 10.0, 1.0, 1.0 def volume_constraint(v): D, L = v return (np.pi / 4) * D**2 * L - V cs_dist = 2.0 res_dist = minimize( lambda v: cs_dist * np.pi * v[0] * v[1] + ct * (np.pi / 2) * v[0] ** 2, x0=[2.0, 2.0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;eq\u0026#34;, \u0026#34;fun\u0026#34;: volume_constraint}, bounds=[(1e-6, None), (1e-6, None)], tol=1e-14, ) print(res_dist.x) [2.94202732 1.47101369] The value published in the original article was \\(D=2.94202734335627\\); it matches up to the 7th decimal place.\nBounds: Limits That Physics Demands # The tank example already used bounds=[(1e-6, None), (1e-6, None)] without explaining why. The reason is more serious than \u0026ldquo;avoiding ugly negative values\u0026rdquo;: without that limit, the problem has no finite minimum. Rewritten as a function of \\(D\\) only (eliminating \\(L\\)), the cost is \\(4c_sV/D + c_t\\frac{\\pi}{2}D^2\\); for negative \\(D\\), the first term plunges without limit as \\(D \\to 0^-\\).\nimport numpy as np from scipy.optimize import minimize V, cs, ct = 10.0, 1.0, 1.0 def cost_2var(v): D, L = v return cs * np.pi * D * L + ct * (np.pi / 2) * D**2 def volume_constraint(v): D, L = v return (np.pi / 4) * D**2 * L - V res_no_bounds = minimize( cost_2var, x0=[-0.5, 40.0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;eq\u0026#34;, \u0026#34;fun\u0026#34;: volume_constraint}, ) print(res_no_bounds.x, res_no_bounds.fun, res_no_bounds.success, res_no_bounds.message) [-8.51038777e-05 1.53038608e+09] -409166.6491516873 False Iteration limit reached Without bounds, the optimizer can \u0026ldquo;find\u0026rdquo; a physical absurdity Starting from a negative diameter, the optimizer does not end up just a little wrong: it chases an ever more negative cost, with L blowing up to more than a billion, and does not even converge (success=False). Redoing it with the same bounds as in the previous example, from the same bad guess:\nimport numpy as np from scipy.optimize import minimize V, cs, ct = 10.0, 1.0, 1.0 def cost_2var(v): D, L = v return cs * np.pi * D * L + ct * (np.pi / 2) * D**2 def volume_constraint(v): D, L = v return (np.pi / 4) * D**2 * L - V res_with_bounds = minimize( cost_2var, x0=[-0.5, 40.0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;eq\u0026#34;, \u0026#34;fun\u0026#34;: volume_constraint}, bounds=[(1e-6, None), (1e-6, None)], tol=1e-12, ) print(res_with_bounds.x, res_with_bounds.success) [2.33508866 2.33508863] True Same guess, completely different result, just because of the bounds. Physical dimensions (diameters, lengths, concentrations, absolute temperatures) almost always call for an explicit lower bound of zero.\nUnconstrained Multivariable Optimization: Rosenbrock # For problems with no constraints at all, scipy.optimize.minimize accepts several methods. The classic benchmark for comparing all of them is the Rosenbrock function (the \u0026ldquo;banana function\u0026rdquo;), \\(f(x,y) = (1-x)^2 + 100(y-x^2)^2\\), with a known minimum at \\((1,1)\\), \\(f=0\\): a narrow, curved valley, purposely designed to be difficult.\nimport numpy as np def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 x_ros = np.linspace(-2, 2, 300) y_ros = np.linspace(-1, 3, 300) X_ros, Y_ros = np.meshgrid(x_ros, y_ros) Z_ros = rosenbrock([X_ros, Y_ros]) Since \\(f\\) has two input variables and one output, it is literally a surface in 3D space: each point \\((x,y)\\) has a \u0026ldquo;height\u0026rdquo; \\(f(x,y)\\). It is worth plotting that surface for real first:\nimport matplotlib.pyplot as plt import numpy as np def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 x_ros = np.linspace(-2, 2, 300) y_ros = np.linspace(-1, 3, 300) X_ros, Y_ros = np.meshgrid(x_ros, y_ros) Z_ros = rosenbrock([X_ros, Y_ros]) fig = plt.figure() ax = fig.add_subplot(projection=\u0026#34;3d\u0026#34;) ax.plot_surface(X_ros, Y_ros, Z_ros, cmap=\u0026#34;viridis\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.set_zlabel(\u0026#34;f(x,y)\u0026#34;) The surface of the Rosenbrock function in 3D. The valley (the darkest band, at the base) barely shows: the vertical scale, which has to accommodate values up to ~2500, squeezes it. The valley is there (it is the darkest band at the base), but the 3D perspective has two problems: part of the surface sits behind another part (occlusion), and the vertical scale, which has to reach \\(\\approx 2500\\) to accommodate the slopes, squeezes exactly the region that matters most. A better way to see the valley is to flatten that surface and look from above, with the third dimension (the height) encoded in a color scale instead of vertical position. That is exactly what a contour map does.\nimport matplotlib.pyplot as plt import numpy as np def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 x_ros = np.linspace(-2, 2, 300) y_ros = np.linspace(-1, 3, 300) X_ros, Y_ros = np.meshgrid(x_ros, y_ros) Z_ros = rosenbrock([X_ros, Y_ros]) fig, ax = plt.subplots() ax.contourf(X_ros, Y_ros, np.log10(Z_ros + 1), levels=50, cmap=\u0026#34;viridis\u0026#34;) ax.plot(-1.2, 1.0, \u0026#34;o\u0026#34;, color=\u0026#34;white\u0026#34;, markeredgecolor=\u0026#34;black\u0026#34;, label=\u0026#34;initial guess\u0026#34;) ax.plot(1, 1, \u0026#34;*\u0026#34;, color=\u0026#34;red\u0026#34;, markersize=16, markeredgecolor=\u0026#34;black\u0026#34;, label=\u0026#34;minimum (1,1)\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.legend() In the viridis color palette, dark purple corresponds to low values of \\(f\\) (near the minimum) and yellow to high values (the slopes). The code uses log10(Z+1), not plain \\(Z\\), in the color scale, for the same reason as in 3D: near the minimum \\(f\\) is almost zero, far from it it exceeds thousands, and on a linear scale that huge difference would make almost the whole plot a single color (that of the edges), erasing the valley again. The logarithm compresses that giant range of values into something where the valley floor and the edges are visible in the same image (the +1 avoids log10(0), which does not exist). The name \u0026ldquo;banana function\u0026rdquo; comes exactly from this shape:\nThe Rosenbrock valley, seen from above: the same surface as the previous 3D plot, now with the height encoded in color, the curved and narrow shape is much clearer. A method that follows the steepest descent direction at each step, with a fixed step size, tends to \u0026ldquo;ricochet\u0026rdquo; from one side to the other inside this narrow valley, instead of walking straight to the minimum. This does not have to stay in words. You can implement steepest descent in five lines and draw the path:\nimport numpy as np def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 def grad_rosenbrock(v): x, y = v return np.array([-2 * (1 - x) - 400 * x * (y - x**2), 200 * (y - x**2)]) fixed_step = 3e-3 v = np.array([-1.2, 1.0]) path = [v.copy()] for _ in range(2000): v = v - fixed_step * grad_rosenbrock(v) path.append(v.copy()) path = np.array(path) side = path[:, 1] - path[:, 0] ** 2 # which side of the valley it is on print(\u0026#34;crossed the valley\u0026#34;, int((np.diff(np.sign(side)) != 0).sum()), \u0026#34;times\u0026#34;) print(f\u0026#34;final f: {rosenbrock(path[-1]):.4f}\u0026#34;) crossed the valley 1071 times final f: 0.3268 Steepest descent with a fixed step: 2000 steps, 1071 valley crossings, and still f = 0.33 instead of 0. The tangle on the right is the method ricocheting between the walls without being able to advance. That is 2000 steps to reach \\(f = 0.33\\), and the method does not even reach the minimum: it gets stuck in a tangle around \\(x = 0.74\\), crossing the valley from side to side more than a thousand times. This is the waste that real methods avoid.\nAnd they do avoid it. The callback parameter of minimize is called at each iteration and lets you record the path that SciPy\u0026rsquo;s methods follow, which, as we will see, does not ricochet:\nimport matplotlib.pyplot as plt import numpy as np from scipy.optimize import minimize def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 x_ros = np.linspace(-2, 2, 300) y_ros = np.linspace(-1, 3, 300) X_ros, Y_ros = np.meshgrid(x_ros, y_ros) Z_ros = rosenbrock([X_ros, Y_ros]) fig, ax = plt.subplots(figsize=(9, 6.5)) ax.contourf(X_ros, Y_ros, np.log10(Z_ros + 1), levels=50, cmap=\u0026#34;viridis\u0026#34;) for method, color in ((\u0026#34;Nelder-Mead\u0026#34;, \u0026#34;C1\u0026#34;), (\u0026#34;BFGS\u0026#34;, \u0026#34;C3\u0026#34;), (\u0026#34;Powell\u0026#34;, \u0026#34;C5\u0026#34;)): path = [np.array([-1.2, 1.0])] res_traj = minimize(rosenbrock, x0=[-1.2, 1.0], method=method, callback=lambda xk, c=path: c.append(np.copy(xk))) traj = np.array(path) ax.plot(traj[:, 0], traj[:, 1], \u0026#34;o-\u0026#34;, color=color, markersize=4, linewidth=1.6, label=f\u0026#34;{method} ({res_traj.nit} iterations, {res_traj.nfev} evaluations)\u0026#34;) ax.plot(1, 1, \u0026#34;*\u0026#34;, color=\u0026#34;white\u0026#34;, markeredgecolor=\u0026#34;black\u0026#34;, markersize=20) ax.legend(loc=\u0026#34;upper left\u0026#34;, fontsize=9) The three SciPy methods inside the same valley. Note the contrast with the previous figure: here there is no ricochet: the three trajectories descend to the bottom of the valley and follow it, almost overlapping. The three trajectories are almost indistinguishable, and that is the result, not a flaw in the figure: all three methods settle the step size before taking each step (that is what a line search does), so each iterate already lands near the valley floor. The ricochet in the previous figure is the price of not doing that.\nNone of the three goes in a straight line. All of them have to follow the curve of the valley, and that is exactly where the iterations are spent: the minimum is a short distance from the initial guess in a straight line, but far in \u0026ldquo;distance traveled inside the valley\u0026rdquo;.\nimport numpy as np from scipy.optimize import minimize def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 for method in [\u0026#34;Nelder-Mead\u0026#34;, \u0026#34;BFGS\u0026#34;, \u0026#34;Powell\u0026#34;]: res = minimize(rosenbrock, x0=[-1.2, 1.0], method=method) print(f\u0026#34;{method:12s} x={np.round(res.x, 5)} f={res.fun:.2e} nit={res.nit} nfev={res.nfev}\u0026#34;) Nelder-Mead x=[1.00002 1.00004] f=8.18e-10 nit=85 nfev=159 BFGS x=[1. 0.99999] f=2.02e-11 nit=32 nfev=117 Powell x=[1. 1.] f=1.79e-26 nit=23 nfev=607 All three get close to \\((1,1)\\), but at very different costs, summed up in two counts that every minimize result carries:\nnit × nfev nit (number of iterations) counts how many times the algorithm repeated its main loop; each iteration typically updates the estimate of the optimal point once. nfev (number of function evaluations) counts how many times \\(f\\) was actually called in total, including all the internal sub-calls of each iteration (line searches, numerical gradient estimation, etc.).\nWhen \\(f\\) is expensive to compute (a simulation, a network call, a heavy model), nfev is usually the metric that really matters, because it measures the actual computational cost. A low nit does not guarantee cheap: here, Powell has the lowest nit (\\(23\\)) but the highest nfev (\\(607\\)), because each of its iterations does a complete line search along several directions, so \u0026ldquo;few iterations\u0026rdquo; does not mean \u0026ldquo;little work\u0026rdquo;.\nNelder-Mead is derivative-free (it does not use the gradient, it only evaluates the function at points and compares), simple and robust, but it needs more iterations in the curved valley (\\(\\approx1.9\\) evaluations per iteration). BFGS uses a numerical approximation of the gradient internally, so it converges in fewer iterations (\\(32\\)), but each one already carries the cost of estimating that gradient (\\(\\approx3.7\\) evaluations per iteration). Powell, also derivative-free, does a thorough line search at each iteration: few iterations, many evaluations, and here that paid off with excellent precision (\\(f \\approx 10^{-26}\\)).\nThe idea behind BFGS The acronym (Broyden, Fletcher, Goldfarb and Shanno) says nothing about the method, and it is worth knowing what it does differently.\nPure gradient descent always walks in the direction of steepest slope. In a narrow, crooked valley like Rosenbrock\u0026rsquo;s, that direction points to the wall of the valley, not to its floor, hence the ricocheting. What would fix this is knowing the curvature, which lives in the Hessian matrix (the second derivatives); with it, Newton\u0026rsquo;s method would take steps that, on a quadratic, would lead straight to the minimum.\nThe problem is that the Hessian of a problem with \\(n\\) variables has \\(n^2\\) entries, and computing it is usually prohibitive. The trick of BFGS is not to compute it: it starts with a rough guess for the Hessian (the identity) and keeps correcting that estimate at each step, using the differences between consecutive gradients and between consecutive positions, information it already has for free. Hence the name of the family: quasi-Newton methods.\nIn most cases, this gives almost the speed of Newton, without the cost of building the Hessian. And that is the reason BFGS is the default of minimize when there are no constraints.\nSupplying the Gradient with jac # The numerical derivatives article promised that this accounting would return: when the analytic gradient is not available, the optimizer estimates it by finite differences, and pays one extra function evaluation per variable, at each iteration. That is where the \\(117\\) evaluations of BFGS above come from, for just \\(32\\) iterations.\nWhen you can write the gradient by hand, the jac parameter avoids that entire cost. For Rosenbrock it takes two lines:\n$$ \\frac{\\partial f}{\\partial x} = -2(1-x) - 400x(y - x^2), \\qquad \\frac{\\partial f}{\\partial y} = 200(y - x^2) $$import numpy as np from scipy.optimize import minimize def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 def jac_rosenbrock(v): x, y = v return np.array([-2 * (1 - x) - 400 * x * (y - x**2), 200 * (y - x**2)]) for label, jac in ((\u0026#34;without jac (finite differences)\u0026#34;, None), (\u0026#34;with analytic jac\u0026#34;, jac_rosenbrock)): r = minimize(rosenbrock, x0=[-1.2, 1.0], method=\u0026#34;BFGS\u0026#34;, jac=jac) print(f\u0026#34;{label:\u0026gt;32}: nit={r.nit:\u0026gt;3} nfev={r.nfev:\u0026gt;4} njev={r.njev:\u0026gt;3} f={r.fun:.2e}\u0026#34;) without jac (finite differences): nit= 32 nfev= 117 njev= 39 f=2.02e-11 with analytic jac: nit= 32 nfev= 39 njev= 39 f=2.54e-15 Two gains at once, and the second is the more interesting:\nThree times fewer function evaluations (117 → 39), with the same number of iterations. The evaluations that vanished were exactly those spent estimating the gradient. Ten thousand times better precision (\\(2\\times10^{-11}\\) → \\(2.5\\times10^{-15}\\)). The gradient estimated by finite differences carries the same \\(U\\) error as in the numerical derivatives article, and that error propagates to the solution. The analytic gradient is exact. 3×\nfewer evaluations\nof the function (117 → 39), with the same number of iterations 10,000×\nmore precision\nin the minimum found, with the analytic gradient When \\(f\\) is expensive (a simulation, an ODE solved at each call, as in the next article), the first gain is the one that matters. When you need many digits, the second.\nMethod Name behind the acronym Uses gradient? When to use Nelder-Mead Nelder and Mead\u0026rsquo;s simplex method No Non-smooth or noisy function; few variables; robustness over speed Powell Powell\u0026rsquo;s method (line searches along conjugate directions) No No gradient available, but a function smooth enough for line searches to pay off; good final precision BFGS Broyden-Fletcher-Goldfarb-Shanno Yes (numerically approximated if not supplied) Smooth, differentiable function; default choice when there are no constraints L-BFGS-B Limited-memory BFGS with Bounds Yes (approximated) Like BFGS, but with bounds, a memory-efficient version, better for many variables SLSQP Sequential Least Squares Programming Yes (approximated) When there are equality/inequality constraints (seen in the tank section) There is no single \u0026ldquo;best method\u0026rdquo;: the choice trades robustness, speed and final precision depending on what is being optimized, and on what really is expensive in the problem.\nLocal × Global Minima # This whole section so far has a silent assumption: that the method converges to the right minimum. That is only guaranteed when there is a single minimum. As soon as the function has more than one, the result depends on where the search starts.\nExample 3: Himmelblau: the result depends on the initial guess Statement: The Himmelblau function, \\(f(x,y) = (x^2+y-11)^2 + (x+y^2-7)^2\\), has four known global minima, all with \\(f=0\\). Minimize it from 5 scattered initial guesses and compare the results.\nSolution:\nimport numpy as np from scipy.optimize import minimize def himmelblau(v): x, y = v return (x**2 + y - 11) ** 2 + (x + y**2 - 7) ** 2 guesses = [(0, 0), (-4, 4), (4, -4), (-4, -4), (5, 5)] for c in guesses: res = minimize(himmelblau, x0=c, method=\u0026#34;BFGS\u0026#34;) print(f\u0026#34;guess={str(c):\u0026lt;8} -\u0026gt; x={str(np.round(res.x, 4)):\u0026lt;17} f={res.fun:.2e}\u0026#34;) guess=(0, 0) -\u0026gt; x=[3. 2.] f=1.38e-13 guess=(-4, 4) -\u0026gt; x=[-2.8051 3.1313] f=1.02e-13 guess=(4, -4) -\u0026gt; x=[ 3.5844 -1.8481] f=6.51e-16 guess=(-4, -4) -\u0026gt; x=[-3.7793 -3.2832] f=1.92e-14 guess=(5, 5) -\u0026gt; x=[3. 2.] f=1.20e-12 Five guesses, four different results (two of them land on the same minimum, \\((3,2)\\)), all equally correct, in the sense that all of them are actual global minima. There is no such thing in the SymPy world: solve on the derivative would return all four at once, without depending on any guess.\nAs in the Rosenbrock case, it is worth seeing the whole surface before flattening it:\nimport matplotlib.pyplot as plt import numpy as np def himmelblau(v): x, y = v return (x**2 + y - 11) ** 2 + (x + y**2 - 7) ** 2 x_grid = np.linspace(-6, 6, 300) y_grid = np.linspace(-6, 6, 300) X, Y = np.meshgrid(x_grid, y_grid) Z = himmelblau([X, Y]) fig = plt.figure() ax = fig.add_subplot(projection=\u0026#34;3d\u0026#34;) ax.plot_surface(X, Y, np.log10(Z + 1), cmap=\u0026#34;viridis\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.set_zlabel(\u0026#34;log10(f+1)\u0026#34;) ax.view_init(elev=60, azim=-45) Once again on a logarithmic scale, for the same reason as before. The four minima show up as four sharp holes in the surface, one at each \u0026ldquo;corner\u0026rdquo; of the relief:\nThe Himmelblau surface, in log: the four minima show up as sharp, symmetric holes. Flattening this same surface into a contour map (now without needing the log, because the viridis color scale already distinguishes the bottom of each basin well even in linear values):\nimport matplotlib.pyplot as plt import numpy as np from scipy.optimize import minimize def himmelblau(v): x, y = v return (x**2 + y - 11) ** 2 + (x + y**2 - 7) ** 2 x_grid = np.linspace(-6, 6, 300) y_grid = np.linspace(-6, 6, 300) X, Y = np.meshgrid(x_grid, y_grid) Z = himmelblau([X, Y]) guesses = [(0, 0), (-4, 4), (4, -4), (-4, -4), (5, 5)] himmelblau_results = [minimize(himmelblau, x0=c, method=\u0026#34;BFGS\u0026#34;).x for c in guesses] fig, ax = plt.subplots() ax.contourf(X, Y, Z, levels=50, cmap=\u0026#34;viridis\u0026#34;) ax.plot([], [], \u0026#34;o\u0026#34;, color=\u0026#34;white\u0026#34;, markeredgecolor=\u0026#34;black\u0026#34;, label=\u0026#34;initial guess\u0026#34;) ax.plot([], [], \u0026#34;*\u0026#34;, color=\u0026#34;red\u0026#34;, markersize=14, markeredgecolor=\u0026#34;black\u0026#34;, label=\u0026#34;result\u0026#34;) for c, xr in zip(guesses, himmelblau_results): ax.plot(*c, \u0026#34;o\u0026#34;, color=\u0026#34;white\u0026#34;, markeredgecolor=\u0026#34;black\u0026#34;) ax.plot(*xr, \u0026#34;*\u0026#34;, color=\u0026#34;red\u0026#34;, markersize=14, markeredgecolor=\u0026#34;black\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.legend() The heat map below shows the four basins of attraction (dark regions) and how each guess (white circle) falls into one of the four minima (red star):\nEach of the four dark basins contains a global minimum: the initial guess decides which one the search ends in. Five guesses make a snapshot. You can turn them into a map: color each point of the plane by the color of the minimum BFGS converges to starting from there. These are the basins of attraction.\nimport numpy as np from scipy.optimize import minimize def himmelblau(v): x, y = v return (x**2 + y - 11) ** 2 + (x + y**2 - 7) ** 2 minima = np.array([[3.0, 2.0], [-2.805118, 3.131312], [-3.779310, -3.283186], [3.584428, -1.848126]]) n_grid = 130 xs_b = ys_b = np.linspace(-6, 6, n_grid) basin = np.full((n_grid, n_grid), -1, dtype=int) for i, yv in enumerate(ys_b): for j, xv in enumerate(xs_b): r = minimize(himmelblau, x0=[xv, yv], method=\u0026#34;BFGS\u0026#34;) d = np.linalg.norm(minima - r.x, axis=1) if d.min() \u0026lt; 1e-3: basin[i, j] = int(np.argmin(d)) for k in range(4): print(f\u0026#34;minimum {tuple(np.round(minima[k], 2).tolist())}: {(basin == k).mean():.1%} of the plane\u0026#34;) minimum (3.0, 2.0): 24.1% of the plane minimum (-2.81, 3.13): 26.1% of the plane minimum (-3.78, -3.28): 24.5% of the plane minimum (3.58, -1.85): 25.3% of the plane Each color marks which minimum BFGS converges to from each point. The four basins split the plane almost equally, but the borders are irregular, with islands of one color inside another\u0026rsquo;s territory. Two readings. The first is reassuring: the four basins get a quarter of the plane each, and every point tested converges to some minimum. The second, not so much: look at the borders. They are neither straight nor smooth: they are irregular, and there are isolated islands of one color right in the middle of another\u0026rsquo;s territory. Near them, two nearly identical initial guesses end up in different minima.\nIt is the same structure that appeared in the Newton basins of the first article in this series, and for the same reason: iterative methods that follow the local geometry cannot guarantee continuity in the final destination.\nHimmelblau has a particularity: its four minima are equally good (all with \\(f=0\\)). The more common problem in practice is another: local minima that are genuinely worse than the global minimum. In that case, every method seen so far (Nelder-Mead, BFGS, Powell, SLSQP) has the same limitation: it starts from a single guess and only sees its neighborhood.\nscipy.optimize.differential_evolution attacks the problem differently. Instead of refining a single estimate step by step, it keeps an entire population of candidates spread across the domain and makes them evolve through generations (combining pairs, applying random mutations, discarding the worst), an evolutionary algorithm, in the same spirit as genetic algorithms. Because it sweeps the entire domain, instead of refining from a single point, it needs no initial guess at all, only the search bounds. The cost is the number of function evaluations, much larger than that of a local method.\nExample 4: One minimum worse than the other Statement: \\(f(x) = x^2 + 10\\sin(x)\\) has only two local minima in the interval \\([-10, 10]\\) (the derivative, \\(f^{\\prime}(x)=2x+10\\cos x\\), only vanishes for \\(|x| \\lesssim 5\\)). Find both and compare with what differential_evolution returns, with no initial guess at all.\nSolution:\nimport numpy as np from scipy.optimize import differential_evolution, minimize def f_two_minima(x): return x**2 + 10 * np.sin(x) for c in [-8, -5, -2, 0, 2, 5, 8]: res = minimize(f_two_minima, x0=[c], method=\u0026#34;BFGS\u0026#34;) print(f\u0026#34;guess={c:5.1f} -\u0026gt; x={res.x[0]:.4f} f={res.fun:.4f}\u0026#34;) res_de = differential_evolution(lambda v: f_two_minima(v[0]), bounds=[(-10, 10)], seed=1) print(\u0026#34;differential_evolution:\u0026#34;, res_de.x, res_de.fun) guess= -8.0 -\u0026gt; x=-1.3064 f=-7.9458 guess= -5.0 -\u0026gt; x=-1.3064 f=-7.9458 guess= -2.0 -\u0026gt; x=-1.3064 f=-7.9458 guess= 0.0 -\u0026gt; x=-1.3064 f=-7.9458 guess= 2.0 -\u0026gt; x=3.8375 f=8.3156 guess= 5.0 -\u0026gt; x=3.8375 f=8.3156 guess= 8.0 -\u0026gt; x=3.8375 f=8.3156 differential_evolution: [-1.30644002] -7.945823375615283 Negative or zero guesses land on the global minimum (\\(f\\approx-7.95\\)); positive guesses get stuck at the worse local minimum (\\(f\\approx 8.32\\): it is not even negative!). differential_evolution finds the global minimum on the first try, without any guess being supplied.\nimport matplotlib.pyplot as plt import numpy as np def f_two_minima(x): return x**2 + 10 * np.sin(x) xs_plot = np.linspace(-10, 10, 400) fig, ax = plt.subplots() ax.plot(xs_plot, f_two_minima(xs_plot)) ax.plot(-1.3064, f_two_minima(-1.3064), \u0026#34;o\u0026#34;, color=\u0026#34;green\u0026#34;, markersize=10, label=\u0026#34;global minimum\u0026#34;) ax.plot(3.8375, f_two_minima(3.8375), \u0026#34;o\u0026#34;, color=\u0026#34;red\u0026#34;, markersize=10, label=\u0026#34;local minimum (worse)\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;f(x)\u0026#34;) ax.legend() The following plot shows the two minima, side by side:\nTwo local minima, very different quality: the one on the right is not even negative. Rastrigin: Where Global Search Stops Being a Luxury # With only two minima, differential_evolution seems overkill: a minimize from two or three guesses would do. The case where it becomes indispensable is another: when the local minima number in the dozens.\nThe Rastrigin function is the canonical benchmark for that. It is a simple paraboloid with a superimposed cosine ripple, which creates a regular grid of holes:\n$$ f(\\mathbf{x}) = 10n + \\sum_{i=1}^{n}\\left[x_i^2 - 10\\cos(2\\pi x_i)\\right] $$The global minimum is trivially known, the origin, with \\(f = 0\\), which makes it perfect for testing whether a method finds it.\nimport numpy as np from scipy.optimize import differential_evolution, minimize def rastrigin(v): return 20 + sum(x**2 - 10 * np.cos(2 * np.pi * x) for x in v) for c in [(-4, -4), (-2, 3), (1, 1), (3, -2), (4.5, 4.5)]: r = minimize(rastrigin, x0=c, method=\u0026#34;BFGS\u0026#34;) print(f\u0026#34;guess {str(c):\u0026gt;11} -\u0026gt; x={np.round(r.x, 3)} f={r.fun:.4f}\u0026#34;) res_rast = differential_evolution(rastrigin, bounds=[(-5.12, 5.12)] * 2, seed=1) print(f\u0026#34;differential_evolution -\u0026gt; x={np.round(res_rast.x, 4)} f={res_rast.fun:.2e}\u0026#34;) guess (-4, -4) -\u0026gt; x=[-3.98 -3.98] f=31.8385 guess (-2, 3) -\u0026gt; x=[-1.99 2.985] f=12.9344 guess (1, 1) -\u0026gt; x=[0.995 0.995] f=1.9899 guess (3, -2) -\u0026gt; x=[ 2.985 -1.99 ] f=12.9344 guess (4.5, 4.5) -\u0026gt; x=[3.98 3.98] f=31.8385 differential_evolution -\u0026gt; x=[-0. -0.] f=0.00e+00 Five guesses, five different local minima, and none of them is the global one. The best BFGS result was \\(f = 1.99\\); the global is \\(0\\). Notice the pattern. The method always stops at the hole closest to the guess: \\((-3.98,\\ -3.98)\\), \\((0.995,\\ 0.995)\\) and so on, all near integers, which is where the Rastrigin holes are.\nThe following figure shows the relief and where each search ended up:\nRastrigin: BFGS barely moves. In four of the five guesses, the result circle fits inside the starting-point square. Only the global search reaches the center. Notice what the figure does not show: any path. In four of the five guesses the result circle is drawn inside the starting square, because the total displacement was \\(0.007\\) to \\(0.029\\) units, on a plot \\(10\\) units wide, less than \\(0.3\\%\\) of the figure. There is no line to draw: BFGS is born inside the hole it is going to die in.\nThe exception is the guess \\((4.5,\\ 4.5)\\), in the upper right corner, the only one that falls between two holes instead of inside one. It travels \\(0.736\\), enough for the eye to see, and even so it only rolls into the neighboring hole, at \\((3.98,\\ 3.98)\\), as far from the global as the others.\ndifferential_evolution hits the global exactly, at the cost of \\(2163\\) function evaluations, against the few dozen of each BFGS.\nThat is the trade: local search is cheap and lies; global search is expensive and does not lie. With a function that is cheap to evaluate, pay for global search. With an expensive function, use local search from several guesses (multistart) and know that you may be in some arbitrary hole.\nPoints of Attention # Never ignore result.success minimize can return a result without having actually converged. Artificially limiting the number of iterations in the Rosenbrock example:\nimport numpy as np from scipy.optimize import minimize def rosenbrock(v): x, y = v return (1 - x) ** 2 + 100 * (y - x**2) ** 2 res_fail = minimize(rosenbrock, x0=[-1.2, 1.0], method=\u0026#34;Nelder-Mead\u0026#34;, options={\u0026#34;maxiter\u0026#34;: 5}) print(res_fail.success, res_fail.message, res_fail.x, res_fail.fun) False Maximum number of iterations has been exceeded. [-1.065 1.1 ] 4.381360062499992 The result, \\((-1.065,\\ 1.1)\\) with \\(f\\approx4.38\\), looks like any number: nothing in it visually warns that it is far from the true minimum, \\((1,1)\\) with \\(f=0\\). Only result.success (and result.message) reveal the problem.\nVariable Scale: the Condition Number # One last point of attention, and the easiest to ignore: the scale of the variables affects how hard the optimizer works. The cause has a name, and it is the same concept that appeared in the polynomials article: ill-conditioning.\nConsider \\(f(x,y) = x^2 + 10^6 y^2\\). It is a perfectly ordinary paraboloid, with its minimum at \\((0,0)\\), but extremely elongated: the curvature in the \\(y\\) direction is a million times greater than in the \\(x\\) direction. The level curves are not circles, they are extremely thin ellipses, and an optimizer that walks \u0026ldquo;downhill\u0026rdquo; ends up ricocheting between the walls of the valley, as in Rosenbrock, only for a different reason.\nimport numpy as np from scipy.optimize import minimize ill_conditioned = lambda v: v[0] ** 2 + 1e6 * v[1] ** 2 well_conditioned = lambda v: v[0] ** 2 + v[1] ** 2 # after rescaling y for method in (\u0026#34;Nelder-Mead\u0026#34;, \u0026#34;BFGS\u0026#34;): r_ill = minimize(ill_conditioned, x0=[1.0, 1.0], method=method) r_well = minimize(well_conditioned, x0=[1.0, 1.0], method=method) print(f\u0026#34;{method:12s} ill-conditioned: nfev={r_ill.nfev:4d} | \u0026#34; f\u0026#34;rescaled: nfev={r_well.nfev:3d}\u0026#34;) Nelder-Mead ill-conditioned: nfev= 134 | rescaled: nfev= 69 BFGS ill-conditioned: nfev= 21 | rescaled: nfev= 9 Both methods spend about twice as many evaluations on the elongated problem, and it is literally the same problem, up to a change of variable \\(y^{\\prime} = 1000y\\). Nothing was gained in precision; you only paid more for it.\nThe condition number of the Hessian measures exactly this elongation: it is the ratio between the largest and the smallest curvature. Here it is \\(10^6\\); in the rescaled problem, it is \\(1\\). The practical rule to take from this:\nNormalize the variables before optimizing If your variables have very different orders of magnitude (a temperature in kelvin (\\(\\sim 10^2\\)) next to a concentration in mol/L (\\(\\sim 10^{-3}\\)), for example), rescale them all to something around 1 before calling the optimizer, and undo the scaling at the end. It is cheap, and is usually the difference between converging quickly and not converging.\nInequality constraint: the wrong sign gives no error at all SLSQP expects an 'ineq' constraint to return a value \\(\\geq 0\\) when satisfied. Flipping that sign by mistake does not raise an exception, it just optimizes the wrong region, silently:\nfrom scipy.optimize import minimize def simple_objective(v): x, y = v return x**2 + y**2 res_correct = minimize( simple_objective, x0=[0, 0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;ineq\u0026#34;, \u0026#34;fun\u0026#34;: lambda v: v[0] + v[1] - 1}, # x+y \u0026gt;= 1 ) res_wrong = minimize( simple_objective, x0=[0, 0], method=\u0026#34;SLSQP\u0026#34;, constraints={\u0026#34;type\u0026#34;: \u0026#34;ineq\u0026#34;, \u0026#34;fun\u0026#34;: lambda v: 1 - (v[0] + v[1])}, # becomes x+y \u0026lt;= 1 ) print(\u0026#34;correct:\u0026#34;, res_correct.x, res_correct.success) print(\u0026#34;flipped sign:\u0026#34;, res_wrong.x, res_wrong.success) correct: [0.5 0.5] True flipped sign: [0. 0.] True Both calls return success=True: the second simply solved \\(x+y\\leq1\\) instead of \\(x+y\\geq1\\), and the minimum of \\(x^2+y^2\\) subject to this swapped constraint is trivially \\((0,0)\\), the origin, which satisfies the inverted constraint with no problem at all. Nothing in the return value warns that the intended constraint was a different one.\nSummary Table # Tool / concept Use Notes minimize_scalar Minimization in 1 variable Only looks for a minimum, never returns a local maximum minimize(..., method=) Multivariable minimization 'Nelder-Mead'/'Powell' (no gradient), 'BFGS' (with approximated gradient) constraints={'type': 'eq'/'ineq', 'fun': ...} Equality/inequality constraints (SLSQP) 'ineq' requires fun(x) \u0026gt;= 0, a flipped sign silently gets the region wrong bounds=[(min, max), ...] Box limits per variable Without them, the optimizer may search a physically absurd domain, or not even converge nit × nfev Algorithm iterations × total function calls nfev is the real cost metric when f is expensive to compute 3D surface → contour map Visualize f(x,y) by flattening the third dimension into color log10(Z+1) in the color scale keeps extreme values from erasing the relief near the minimum Local × global minimum Result depends on the initial guess when there is more than one minimum SymPy\u0026rsquo;s solve finds all of them at once; minimize only the one closest to the guess differential_evolution Global search, needs no initial guess More computationally expensive; useful when there are multiple minima of different quality result.success / result.message Explicit confirmation of convergence Never assume a minimize converged without checking Variable scale Affects the number of evaluations until convergence Normalizing to comparable orders of magnitude speeds up the search Next Steps # This article covered the fundamentals of scipy.optimize with a single truly real problem, the cylindrical tank, generalizing what the SymPy optimization article solved symbolically. But real optimization rarely stops at a single equality constraint in two variables: chemistry, physics and engineering bring problems with inequality constraints, more elaborate cost functions, and situations in which a system\u0026rsquo;s own differential equation is part of what you want to optimize. In the next article, those real applications come into play, including one that directly reuses the solve_ivp from this series\u0026rsquo; differential equations article.\n","date":"2026-10-28","externalUrl":null,"permalink":"/2026/10/optimization-scipy/","section":"Blog","summary":"Differentiating and setting the result to zero solves textbook optimization. With several variables, constraints, or a function without a tractable derivative, that calculation breaks down. We will see minimize, SLSQP and bounds from scipy.optimize, local minima, and global search with differential_evolution.","title":"Optimization with SciPy","type":"posts"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/tags/otimizacao/","section":"tags","summary":"","title":"otimização","type":"tags"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/categories/programacao/","section":"Categorias","summary":"","title":"Programação","type":"categories"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/categories/programming/","section":"Categories","summary":"","title":"Programming","type":"categories"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/tags/python/","section":"tags","summary":"","title":"Python","type":"tags"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/tags/scipy/","section":"tags","summary":"","title":"SciPy","type":"tags"},{"content":"Some articles are organized into series, which are collections of articles related to a specific theme. A series can include articles from different categories and is a way of grouping content that shares a common focus. The articles in a series are presented in a specific order, which can help guide the reader through a more complex or wide-ranging topic.\n","date":"2026-10-28","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"2026-10-28","externalUrl":null,"permalink":"/tags/","section":"tags","summary":"","title":"tags","type":"tags"},{"content":"","date":"2026-10-21","externalUrl":null,"permalink":"/tags/calculo/","section":"tags","summary":"","title":"cálculo","type":"tags"},{"content":"","date":"2026-10-21","externalUrl":null,"permalink":"/tags/calculus/","section":"tags","summary":"","title":"calculus","type":"tags"},{"content":" The formula for the period of a simple pendulum, \\(T = 2\\pi\\sqrt{L/g}\\), is one of the first physics equations every student memorizes, and almost no one notices that it only holds for small oscillations. A real pendulum, released from a large angle, has a longer period than this formula predicts. And the exact differential equation of this pendulum, without the small-angle simplification, has no closed-form solution at all, not even SymPy can solve it.\nThis is not a limitation of SymPy specifically. A good share of the ordinary differential equations (ODEs) that describe real systems (nonlinear, with more than one variable, with resistance or reaction terms) simply have no analytic solution. When that happens, one way out remains: solve numerically.\nThis article closes the numerical calculus block of the series. The numerical derivatives article showed how to differentiate data with no formula behind it; the previous one integrated functions and tables, but always with the integrand known beforehand. Here that assumption falls: when the rate of change depends on the current value of the quantity itself, there is no integrand to tabulate before starting, because each step depends on the previous one. That is the difference between computing an area and solving a differential equation.\nWhy Does Solving Differential Equations Numerically Matter? # Physics: large-angle pendulums, perturbed orbits, projectiles with air resistance. Most real mechanical systems only have a closed-form solution under simplifications that do not always hold. Chemistry and reaction kinetics: mechanisms with more than one step (consecutive, reversible, catalyzed reactions) give rise to systems of coupled differential equations that rarely close analytically. Electrical and control engineering: circuits with multiple components, feedback control systems. Time-response analysis depends on solving ODEs, often numerically. Biology and epidemiology: models of population growth and disease spread (predator-prey, SIR) are nonlinear by nature and almost always require a numerical solution. From dsolve to solve_ivp # The SymPy derivatives article solved a single differential equation: \\(f(x) - f^{\\prime}(x) = 0\\), whose symbolic solution is the family \\(f(x) = C_1 e^x\\), with the constant \\(C_1\\) left open. That is how dsolve always answers: the general solution, valid for any initial condition.\nimport sympy as sp t = sp.symbols(\u0026#34;t\u0026#34;) f = sp.Function(\u0026#34;f\u0026#34;) diff_eq = sp.Eq(f(t) - sp.diff(f(t), t), 0) print(sp.dsolve(diff_eq, f(t))) Eq(f(t), C1*exp(t)) scipy.integrate.solve_ivp solves the same kind of problem in a different way: instead of a family of solutions, it returns a numerical solution, specific to one concrete initial condition, hence the name, initial value problem (IVP). The signature convention is always the same: a function field(t, y) that returns the derivative of each component of y, a time interval, and the initial condition:\nimport numpy as np from scipy.integrate import solve_ivp def exponential_field(t, y): return y # f\u0026#39;(t) = f(t) sol = solve_ivp(exponential_field, [0, 3], [1.0], dense_output=True, rtol=1e-10, atol=1e-12) t_plot = np.linspace(0, 3, 200) y_numeric = sol.sol(t_plot)[0] y_analytic = np.exp(t_plot) print(\u0026#34;maximum error numeric × analytic:\u0026#34;, np.max(np.abs(y_numeric - y_analytic))) maximum error numeric × analytic: 1.581241804160527e-09 With \\(f(0) = 1\\) fixing the constant (\\(C_1 = 1\\)), the numerical curve falls right on top of the analytic curve \\(e^t\\), with an error on the order of \\(10^{-9}\\), a pure artifact of numerical tolerance, not of the technique. dense_output=True makes it possible to evaluate the solution at any point of the interval after solving, which is what sol.sol(t_plot) does here.\nBut two parameters showed up without explanation, rtol and atol, and they will reappear in almost every call in this article. It is worth understanding what they mean before moving on.\nrtol and atol: how solve_ivp decides whether a step is good Every adaptive step-size method needs a criterion to accept or reject a step. solve_ivp estimates the error made in each step and compares it with a limit built from the two parameters: $$\\text{estimated error} \\le \\texttt{atol} + \\texttt{rtol} \\times |y|$$ If the error exceeds this limit, the step is rejected and redone with a smaller \\(h\\); if it stays well below, the integrator allows itself to increase \\(h\\) on the next step. That is how it chooses the step size on its own.\nThe two play different roles, which is why both exist:\nrtol (relative tolerance) controls the significant digits. An rtol=1e-6 asks, roughly, for 6 correct digits, regardless of whether \\(y\\) is \\(10^{-3}\\) or \\(10^{8}\\). atol (absolute tolerance) is a floor, and it exists for the case where \\(y\\) passes near zero. There the term \\(\\texttt{rtol} \\times |y|\\) also tends to zero, and demanding relative precision on a tiny number would be impossible: the integrator would keep shrinking the step forever. atol says \u0026ldquo;below this absolute value, stop worrying\u0026rdquo;. The default values are rtol=1e-3 and atol=1e-6, the same for all methods, and quite loose: they were chosen to be fast, not precise. The difference is easy to measure, solving the same exponential both ways:\nimport numpy as np from scipy.integrate import solve_ivp def exponential_field(t, y): return y # f\u0026#39;(t) = f(t) t_plot = np.linspace(0, 3, 200) y_analytic = np.exp(t_plot) sol_default = solve_ivp(exponential_field, [0, 3], [1.0], dense_output=True) print(\u0026#34;error with the defaults:\u0026#34;, np.max(np.abs(sol_default.sol(t_plot)[0] - y_analytic))) error with the defaults: 0.0020342038563727627 An error of \\(2 \\times 10^{-3}\\) against the \\(1.6 \\times 10^{-9}\\) from before: a million times larger, for the same ODE and the same interval. Three correct digits are enough to draw a plot, but not to compare against an analytic solution, which is what we will be doing here all the time. That is why the calls in this article almost always tighten both.\n\u0026ldquo;Almost always\u0026rdquo;: the stiffness comparison, later on, deliberately uses the defaults, because the step counts of each method only make sense if all of them run under the same criterion, and the default is the criterion most people will use in practice.\nimport matplotlib.pyplot as plt import numpy as np from scipy.integrate import solve_ivp def exponential_field(t, y): return y # f\u0026#39;(t) = f(t) sol = solve_ivp(exponential_field, [0, 3], [1.0], dense_output=True, rtol=1e-10, atol=1e-12) t_plot = np.linspace(0, 3, 200) y_numeric = sol.sol(t_plot)[0] y_analytic = np.exp(t_plot) fig, ax = plt.subplots() ax.plot(t_plot, y_analytic, label=r\u0026#34;analytic $e^t$\u0026#34;) ax.plot(t_plot, y_numeric, \u0026#34;--\u0026#34;, label=\u0026#34;solve_ivp\u0026#34;) ax.set_xlabel(\u0026#34;t\u0026#34;) ax.set_ylabel(\u0026#34;f(t)\u0026#34;) ax.legend() The following plot shows the two curves overlaid:\nf\u0026rsquo;(t) = f(t), f(0) = 1: the numerical solution falls exactly on the analytic one. The Pendulum That the High School Formula Does Not Describe # Back to the pendulum from the opening: the exact equation of motion, with no approximation at all, is\n$$ \\theta''(t) + \\frac{g}{L}\\sin\\theta(t) = 0 $$The \\(\\sin\\theta\\) term is the problem: it is nonlinear, and it is exactly what prevents a closed-form solution in terms of elementary functions. The exact solution exists, but it is written with elliptic integrals, which are not elementary functions; we will come back to them shortly, to check the period we will measure numerically. The high school simplification assumes a small angle, \\(\\sin\\theta \\approx \\theta\\), which turns the equation into a linear ODE with a closed-form solution in cosine: \\(\\theta(t) = \\theta_0 \\cos(\\omega t)\\), with \\(\\omega = \\sqrt{g/L}\\), hence the period formula, \\(T = 2\\pi/\\omega = 2\\pi\\sqrt{L/g}\\).\nsolve_ivp needs none of these simplifications, but it only solves first-order systems, and the pendulum equation is second order. The conversion is mechanical, and it is worth doing explicitly, because it is the maneuver that repeats for every high-order ODE:\nReducing the order: from one 2nd-order equation to two 1st-order ones Name each derivative up to one order below the maximum as a new variable: $$u_1 = \\theta, \\qquad u_2 = \\dot\\theta$$ Now write the derivative of each one. The first is immediate, by definition; the second comes from the original equation, isolating \\(\\ddot\\theta\\): $$\\dot u_1 = u_2, \\qquad \\dot u_2 = -\\frac{g}{L}\\sin u_1$$ Done: two first-order equations, each giving the derivative of one component as a function of both. That is exactly what the function field(t, y) needs to return, in the order [u1_dot, u2_dot].\nThe recipe holds in general: an equation of order \\(n\\) becomes a system of \\(n\\) first-order equations, with \\(n\\) initial conditions. A damped oscillator, an RLC circuit, a mass-spring system: all go through here.\nTranslating to code, \\(\\theta\\) and \\(\\dot\\theta\\) become the two components of y:\nimport numpy as np from scipy.integrate import solve_ivp g = 9.81 # m/s^2 L = 1.0 # m omega = np.sqrt(g / L) def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] def linear_pendulum(t, theta0): return theta0 * np.cos(omega * t) t_span = [0, 6] t_eval = np.linspace(*t_span, 400) for theta0_deg in [10, 60]: theta0 = np.radians(theta0_deg) sol = solve_ivp(nonlinear_pendulum, t_span, [theta0, 0.0], t_eval=t_eval, rtol=1e-8, atol=1e-10) diff_max = np.max(np.abs(np.degrees(sol.y[0] - linear_pendulum(t_eval, theta0)))) print(f\u0026#34;theta0={theta0_deg} degrees: maximum divergence from the linear approximation = {diff_max:.3f} degrees\u0026#34;) theta0=10 degrees: maximum divergence from the linear approximation = 0.330 degrees theta0=60 degrees: maximum divergence from the linear approximation = 69.384 degrees For \\(10°\\), the small-angle approximation is off by less than half a degree, which is why it works so well in textbook problems, which almost always use small angles. For \\(60°\\), the divergence exceeds \\(69°\\).\nBut be careful about what this number means. It is tempting to read \u0026ldquo;\\(69°\\) of divergence\u0026rdquo; as \u0026ldquo;the approximation gets the shape of the motion wrong\u0026rdquo;, but that is not it. At \\(60°\\), the nonlinear solution still looks a lot like a sinusoid; what changes is the period. And since the comparison above runs for \\(6\\) s, that is, almost three oscillations, a small difference in period accumulates until the two curves are out of phase. The \\(69°\\) measure accumulated phase drift, not shape error.\nMeasuring the Real Period # The hook that opened this article was exactly about this: a pendulum released from a large angle has a longer period than \\(2\\pi\\sqrt{L/g}\\). Time to put a number on it, and the right tool is the events parameter of solve_ivp, which stops (or just records) the integration when a function of the state crosses zero.\nHere, the pendulum starts from rest, so the angular velocity \\(\\dot\\theta\\) returns to zero exactly at each extreme of the oscillation. Detecting this crossing gives half a period:\nimport numpy as np from scipy.integrate import solve_ivp g = 9.81 # m/s^2 L = 1.0 # m def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] def zero_velocity(t, y): return y[1] # theta dot zero_velocity.direction = 1 # only negative-to-positive crossings T0 = 2 * np.pi * np.sqrt(L / g) for theta0_deg in [10, 60, 90, 170]: theta0 = np.radians(theta0_deg) sol = solve_ivp(nonlinear_pendulum, [1e-9, 20], [theta0, 0.0], events=zero_velocity, rtol=1e-11, atol=1e-13) T = 2 * sol.t_events[0][0] print(f\u0026#34;theta0={theta0_deg:4d}° T={T:.4f} s T/T0={T/T0:.4f}\u0026#34;) theta0= 10° T=2.0099 s T/T0=1.0019 theta0= 60° T=2.1529 s T/T0=1.0732 theta0= 90° T=2.3678 s T/T0=1.1803 theta0= 170° T=4.8935 s T/T0=2.4394 Now the hook is paid off, with interpretable numbers. At \\(10°\\), the real period exceeds the high school formula by \\(0.2\\%\\), invisible in any laboratory experiment. At \\(60°\\), it is already \\(7.3\\%\\). At \\(170°\\), almost hanging upward, the pendulum takes more than twice the predicted time.\nChecking against the elliptic integral The exact solution of the pendulum exists, but in terms of a complete elliptic integral of the first kind, \\(K(m)\\), which is not an elementary function, and that is why dsolve does not close it: $$\\frac{T}{T_0} = \\frac{2}{\\pi} K\\left(\\sin^2\\frac{\\theta_0}{2}\\right)$$ scipy.special.ellipk computes \\(K\\), so we can check the numerical measurement against the exact value:\nimport numpy as np from scipy.integrate import solve_ivp from scipy.special import ellipk g = 9.81 # m/s^2 L = 1.0 # m def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] def zero_velocity(t, y): return y[1] # theta dot zero_velocity.direction = 1 # only negative-to-positive crossings T0 = 2 * np.pi * np.sqrt(L / g) print(f\u0026#34;{\u0026#39;theta0\u0026#39;:\u0026gt;7} {\u0026#39;numeric T\u0026#39;:\u0026gt;11} {\u0026#39;T/T0\u0026#39;:\u0026gt;7} {\u0026#39;(2/pi)K\u0026#39;:\u0026gt;8} {\u0026#39;difference\u0026#39;:\u0026gt;10}\u0026#34;) for theta0_deg in [10, 60, 90, 170]: theta0 = np.radians(theta0_deg) sol = solve_ivp(nonlinear_pendulum, [1e-9, 20], [theta0, 0.0], events=zero_velocity, rtol=1e-11, atol=1e-13) T = 2 * sol.t_events[0][0] exact = (2 / np.pi) * ellipk(np.sin(theta0 / 2) ** 2) print(f\u0026#34;{theta0_deg:\u0026gt;6}° {T:\u0026gt;11.5f} {T/T0:\u0026gt;7.5f} {exact:\u0026gt;8.5f} {abs(T/T0 - exact):\u0026gt;10.2e}\u0026#34;) theta0 numeric T T/T0 (2/pi)K difference 10° 2.00989 1.00191 1.00191 9.97e-10 60° 2.15287 1.07318 1.07318 9.96e-10 90° 2.36784 1.18034 1.18034 9.95e-10 170° 4.89352 2.43936 2.43936 9.22e-10 Agreement to the ninth decimal place in all four rows, and the residual is the event\u0026rsquo;s own stopping criterion, not an error of the method.\nA convention trap: there are two notations for the elliptic integral in the literature. Some write \\(K(k)\\), with the modulus \\(k\\); others, \\(K(m)\\), with the parameter \\(m = k^2\\). SciPy uses the second, hence the np.sin(theta0/2) ** 2, with the square. Passing \\(k\\) instead of \\(m\\) raises no error at all, only a silently wrong answer:\nimport numpy as np from scipy.special import ellipk theta0 = np.radians(60) print(\u0026#34;with m = sin²(θ₀/2), correct:\u0026#34;, (2 / np.pi) * ellipk(np.sin(theta0 / 2) ** 2)) print(\u0026#34;with k = sin(θ₀/2), wrong :\u0026#34;, (2 / np.pi) * ellipk(np.sin(theta0 / 2))) with m = sin²(θ₀/2), correct: 1.0731820071493645 with k = sin(θ₀/2), wrong : 1.1803405990160962 Notice the wrong value: \\(1.1803\\) is exactly the right answer for \\(\\theta_0 = 90°\\). In other words, the mistake returns a perfectly plausible number, the kind that would go unnoticed in a review; that is why the check against the numerical solution, done above, is worth it even when an exact formula exists.\nThere is also the series approximation that many books give, \\(T/T_0 \\approx 1 + \\theta_0^2/16\\). It gives \\(1.0019\\) at \\(10°\\) and \\(1.0685\\) at \\(60°\\), good in the first two cases. But it predicts \\(1.55\\) at \\(170°\\), against the real \\(2.44\\): an error of \\(36\\%\\). The progressive degradation of this series is, in itself, a good lesson on how far an approximation can be stretched.\nSweeping the amplitude from \\(1°\\) to \\(179°\\), the three descriptions line up side by side:\nThe real period diverges from the formula T = 2π√(L/g) as the amplitude grows, and shoots up near 180°. The series approximation follows well up to about 60° and then gets lost. The dotted line at \\(1.0\\) is what the high school formula predicts: that the period does not depend on the amplitude. It is a good description up to about \\(20°\\), and that is what makes a pendulum clock possible, as long as it swings little.\nAnd the divergence near \\(180°\\) has a direct physical meaning: at that amplitude the pendulum reaches the top with zero velocity, in (unstable) equilibrium. It would take an infinite time to complete the oscillation.\nPhase Portrait: All the Solutions at Once # Each call to solve_ivp returns one curve, for one initial condition. The phase portrait shows the whole set: instead of \\(\\theta\\) against time, one plots \\(\\dot\\theta\\) against \\(\\theta\\), and each initial condition becomes a closed or open trajectory in this plane.\nPhase portrait of the pendulum. Inside the separatrix (red), the trajectories are closed: the pendulum oscillates. Outside it, they are open: it has enough energy to make full turns. The figure summarizes the entire dynamics in one image:\nThe point at the center is the stable equilibrium (pendulum at rest at the bottom). Near it, the curves are almost ellipses: this is the linear regime, the high school one. The closed curves are oscillations. The larger the orbit, the larger the amplitude, and, as we just saw, the longer the period. The two \u0026ldquo;X\u0026rdquo; marks at \\(\\pm180°\\) are the unstable equilibrium: the upright pendulum. They are the same physical point, since \\(-180°\\) and \\(+180°\\) coincide. The red curve is the separatrix: the trajectory with exactly the energy needed to reach the top, and none to spare. It is the boundary between the two behaviors, and it is what the divergence of the period at \\(180°\\) corresponds to. The open curves, above, are rotations: enough energy to pass the top and keep spinning, always in the same direction. Notice that none of this is visible in a single solution in time. That is why the phase portrait is the standard tool for understanding nonlinear systems: it shows what can happen, not just what happened with one specific initial condition.\nimport matplotlib.pyplot as plt import numpy as np from scipy.integrate import solve_ivp g = 9.81 # m/s^2 L = 1.0 # m def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] omega = np.sqrt(g / L) def linear_pendulum(t, theta0): return theta0 * np.cos(omega * t) theta0 = np.radians(60) # 2 periods of the NONLINEAR pendulum (T = 2.153 s), instead of a fixed 6 s: this way # the phase lag grows visibly without turning into a tangle of curves t_span_fig = [0, 2 * 2.1529] t_eval_fig = np.linspace(*t_span_fig, 400) sol_pendulum = solve_ivp(nonlinear_pendulum, t_span_fig, [theta0, 0.0], t_eval=t_eval_fig, rtol=1e-8, atol=1e-10) fig, ax = plt.subplots() ax.plot(t_eval_fig, np.degrees(sol_pendulum.y[0]), label=\u0026#34;nonlinear (solve_ivp)\u0026#34;) ax.plot(t_eval_fig, np.degrees(linear_pendulum(t_eval_fig, theta0)), \u0026#34;--\u0026#34;, label=r\u0026#34;linear ($\\sin\\theta\\approx\\theta$)\u0026#34;) ax.set_xlabel(\u0026#34;t (s)\u0026#34;) ax.set_ylabel(r\u0026#34;$\\theta$ (degrees)\u0026#34;) ax.legend() The following plot makes this visible: the two curves start together, but quickly come apart, because the nonlinear one has a longer period (it oscillates more slowly) than the linear approximation:\nTwo periods of the pendulum at 60°. Notice that the amplitudes are identical and both curves still look like sinusoids: what changes is the period, and the phase lag accumulates oscillation after oscillation. Choosing a Method: When the Problem Is Stiff # solve_ivp accepts several methods via method=, and the right choice can be the difference between solving in milliseconds or taking much longer to solve the same equation.\nWhat \u0026ldquo;Stiffness\u0026rdquo; Means # A classic example exposes the problem: \\(y^{\\prime} = -1000(y - \\cos t)\\). This equation embeds two very different time scales. The factor \\(-1000\\) makes any deviation between \\(y\\) and \\(\\cos t\\) vanish almost instantly: the time scale of this relaxation is \\(1/1000 = 0.001\\) s. The \\(\\cos t\\) itself, which \\(y\\) starts to follow closely after this initial transient, varies on a much slower scale (period of \\(2\\pi \\approx 6.3\\) s). A system like this, with one component that reacts extremely fast and another that evolves slowly, is called stiff.\nThe problem is not lack of precision, it is stability: a method like RK45 (seen next) chooses the step size based on the fastest dynamics of the system, and keeps needing tiny steps even after that fast dynamics has already \u0026ldquo;died\u0026rdquo; and \\(y\\) is already evolving smoothly along with \\(\\cos t\\). If the step exceeds the limit imposed by the fast scale, the method does not just become imprecise: it diverges numerically (the solution oscillates and blows up). That is why the integrator is forced to keep small steps the whole time, even in the \u0026ldquo;easy\u0026rdquo; part of the solution, hence the cost.\nExplicit and Implicit Methods, Step by Step # Every numerical method for ODEs works in the same general way: given the current value, \\(y_n\\), compute the next one, \\(y_{n+1}\\), after a step of size \\(h\\). The difference between an explicit and an implicit method is in how that computation is done.\nExplicit method × implicit method A method is explicit when it computes \\(y_{n+1}\\) directly, using only what is already known: the current value and the derivative at it (or at intermediate points already computed from it). The simplest case, the explicit Euler method: $$y_{n+1} = y_n + h \\cdot f(t_n, y_n)$$ Everything on the right-hand side is already a known number before the step: just substitute and compute.\nA method is implicit when \\(y_{n+1}\\) appears on both sides of the equation, including inside the very function being integrated. The implicit equivalent of the Euler method above: $$y_{n+1} = y_n + h \\cdot f(t_{n+1}, y_{n+1})$$ Since \\(y_{n+1}\\) is at the same time what we want to find and one of the arguments of \\(f\\) on the right-hand side, it cannot be isolated in a direct computation: solving this equation normally requires an iterative method (such as Newton\u0026rsquo;s) at each step.\nThe Runge-Kutta methods (RK45, RK23, DOP853, seen in the table below) follow the explicit logic, only combining several intermediate evaluations of \\(f\\) in a more precise formula, but always a direct computation, without having to solve any equation. The extra cost per step of an implicit method buys stability: its stability region is much larger, and does not depend on the fast scale of the system the way an explicit method\u0026rsquo;s does; that is why it tolerates much larger steps on stiff problems, like the one in the example above.\nimport numpy as np from scipy.integrate import solve_ivp def stiff(t, y): return [-1000 * (y[0] - np.cos(t))] for method_name in [\u0026#34;RK45\u0026#34;, \u0026#34;Radau\u0026#34;, \u0026#34;LSODA\u0026#34;]: sol = solve_ivp(stiff, [0, 2], [0.0], method=method_name) print(f\u0026#34;{method_name:\u0026gt;6}: success={sol.success} steps={len(sol.t)} nfev={sol.nfev}\u0026#34;) RK45: success=True steps=611 nfev=4238 Radau: success=True steps=15 nfev=100 LSODA: success=True steps=58 nfev=106 RK45 needed \\(611\\) steps and \\(4238\\) function evaluations to solve the same problem that Radau solves in \\(15\\) steps and \\(100\\) evaluations, more than 40 times less work. That is the practical symptom of stiffness: a number of steps or evaluations much larger than expected for the default explicit method, on a problem that \u0026ldquo;does not look\u0026rdquo; that complicated.\n4,238\nRK45\nfunction evaluations, in 611 steps 100\nRadau\nevaluations for the same problem, in 15 steps 40×\nless work\nthe practical symptom of a stiff problem But these counts say how much, not where. Plotting the size of each step along the integration, the mechanism shows up:\nimport matplotlib.pyplot as plt import numpy as np from scipy.integrate import solve_ivp def stiff(t, y): return [-1000 * (y[0] - np.cos(t))] fig, ax = plt.subplots(figsize=(9, 5.5)) for method_name, color in ((\u0026#34;RK45\u0026#34;, \u0026#34;C1\u0026#34;), (\u0026#34;Radau\u0026#34;, \u0026#34;C0\u0026#34;), (\u0026#34;LSODA\u0026#34;, \u0026#34;C3\u0026#34;)): sol_p = solve_ivp(stiff, [0, 2], [0.0], method=method_name) steps = np.diff(sol_p.t) # the interval between consecutive instants ax.semilogy(sol_p.t[1:], steps, \u0026#34;o-\u0026#34;, color=color, markersize=3, linewidth=1, label=f\u0026#34;{method_name} ({len(sol_p.t)} steps)\u0026#34;) print(f\u0026#34;{method_name:\u0026gt;6}: median step={np.median(steps):.2e}\u0026#34; f\u0026#34; maximum={steps.max():.2e}\u0026#34;) ax.axhline(1 / 1000, color=\u0026#34;C4\u0026#34;, linestyle=\u0026#34;--\u0026#34;, linewidth=1.5, label=\u0026#34;fast time scale (1/1000 s)\u0026#34;) ax.set_xlabel(\u0026#34;t (s)\u0026#34;) ax.set_ylabel(\u0026#34;step size h\u0026#34;) ax.legend(fontsize=9) RK45 never manages to increase the step: it stays stuck near the fast time scale of the problem throughout the integration. The implicit methods escape it as soon as the initial transient passes. RK45: median step=3.25e-03 maximum=4.18e-03 Radau: median step=1.43e-03 maximum=1.67e+00 LSODA: median step=5.75e-04 maximum=1.75e-01 The maximum step tells the whole story. RK45 never goes past \\(4.2\\times10^{-3}\\), always in the same order of magnitude as the fast time scale of the problem (\\(1/1000\\) s), from start to finish. Radau reaches \\(1.67\\), four hundred times larger.\nAnd that is what stiffness means in practice. After the initial transient, the solution is smooth: \\(y\\) simply follows \\(\\cos t\\), and a large step would deliver the precision without any problem. But the explicit method cannot increase the step: not for lack of precision, but because above a certain \\(h\\) it becomes unstable and the solution blows up. It remains hostage to a fast dynamics that no longer exists.\nWhat \u0026ldquo;Order p(q)\u0026rdquo; Means # Before the table, it is worth understanding the \u0026ldquo;order \\(p(q)\\)\u0026rdquo; notation that appears in almost every method. It describes an embedded pair of formulas, which run together reusing the same intermediate evaluations of \\(f\\):\nThe solution that actually advances, \\(y_{n+1}\\), comes from the formula of order \\(p\\) (the higher one). \u0026ldquo;Order \\(p\\)\u0026rdquo; means that the error accumulated over the integration falls like \\(O(h^p)\\), the same concept of order already seen in the numerical derivatives article with finite differences (the central one was \\(O(h^2)\\), the forward/backward, \\(O(h)\\)): the higher the order, the faster the error falls as \\(h\\) decreases. The formula of order \\(q\\) (the lower one, in parentheses) does not advance the solution: it serves only to estimate the step error, comparing its result with that of the formula of order \\(p\\). Since the two formulas share the evaluations of \\(f\\), this estimate comes at no extra cost, and it is what allows solve_ivp to grow or shrink \\(h\\) on its own along the integration (adaptive step size), without the user having to fix a value. So \u0026ldquo;RK45, order 5(4)\u0026rdquo; reads: it advances with the 5th-order formula and measures the step error with a 4th-order formula embedded in it.\nQuick Guide to Methods # The following table summarizes the options of solve_ivp, with the full name behind each acronym and when each one is justified:\nMethod Name behind the acronym Type When to use RK45 (default) Runge-Kutta, Dormand-Prince pair of formulas, order 5(4) Explicit First choice for most non-stiff problems, a good balance between cost and precision RK23 Runge-Kutta, Bogacki-Shampine pair of formulas, order 3(2) Explicit Low precision is acceptable, or the field(t, y) is expensive to evaluate and fewer evaluations per step matter more than the order DOP853 Dormand-Prince Runge-Kutta, order 8 Explicit Smooth, non-stiff problem where high precision is needed, more accurate than RK45, at the cost of more evaluations per step Radau Implicit Runge-Kutta of the Radau IIA family, order 5(3) Implicit Stiff problems that also need high precision BDF Backward Differentiation Formulas, variable order from 1 to 5 Implicit Stiff problems with many variables, robust and efficient on large systems LSODA Solver from the ODEPACK package (Lawrence Livermore National Laboratory), automatically switching between an Adams-type method (non-stiff) and BDF Automatic When it is not known beforehand whether the problem is stiff, it detects the transition and switches method on its own (The full names and the origin of each method come from the SciPy documentation itself, which cites the original papers for each one.)\nWithout this table as a reference, the reasonable default choice is always RK45; it is only worth switching when the symptom of stiffness appears (steps or nfev well above what is expected) or when the precision of RK45 is not enough for a problem known to be smooth.\nReal Applications # With the basic mechanics and the choice of method established, five applications follow, from the simplest (with a closed-form solution, for validation) to the ones that do not close at all.\nExample 1: Logistic population growth Statement: A population grows according to the logistic model, \\(\\frac{dP}{dt} = rP\\left(1 - \\frac{P}{K}\\right)\\), with growth rate \\(r = 0.5\\) year\\(^{-1}\\), carrying capacity \\(K=100\\), and initial population \\(P_0=5\\). Solve numerically and compare with the known analytic solution.\nSolution: The closed-form solution of this model is \\(P(t) = \\dfrac{K}{1 + (K/P_0 - 1)e^{-rt}}\\):\nimport numpy as np from scipy.integrate import solve_ivp r, K, P0 = 0.5, 100.0, 5.0 def logistic(t, P): return [r * P[0] * (1 - P[0] / K)] def logistic_analytic(t): return K / (1 + (K / P0 - 1) * np.exp(-r * t)) t_eval_log = np.linspace(0, 20, 200) sol_log = solve_ivp(logistic, [0, 20], [P0], t_eval=t_eval_log, rtol=1e-8, atol=1e-10) error = np.max(np.abs(sol_log.y[0] - logistic_analytic(t_eval_log))) print(\u0026#34;maximum error:\u0026#34;, error) maximum error: 5.216148338149651e-06 An error on the order of \\(10^{-6}\\): the nonlinear ODE is solved numerically without the model having to be linearized or simplified in any way, and the result matches the closed-form formula within the requested tolerance.\nExample 2: Lotka-Volterra: predator and prey Statement: The Lotka-Volterra predator-prey model describes two coupled populations: $$\\frac{dP}{dt} = \\alpha P - \\beta P Q, \\qquad \\frac{dQ}{dt} = \\delta P Q - \\gamma Q$$ The prey \\(P\\) grows on its own (\\(\\alpha\\)) and is eaten (\\(\\beta PQ\\)); the predator \\(Q\\) dies on its own (\\(\\gamma\\)) and reproduces by eating (\\(\\delta PQ\\)). Solve with \\(\\alpha=1\\), \\(\\beta=0.02\\), \\(\\delta=0.01\\), \\(\\gamma=0.5\\), starting from 40 prey and 9 predators.\nSolution: This is the first example in the article with parameters passed via args, instead of captured from the global scope, which is the recommended way, and avoids the class of bug in which changing a global variable silently changes the result of another cell:\nimport numpy as np from scipy.integrate import solve_ivp def lotka_volterra(t, y, alpha, beta, delta, gamma): prey, predator = y return [alpha * prey - beta * prey * predator, delta * prey * predator - gamma * predator] params = (1.0, 0.02, 0.01, 0.5) # alpha, beta, delta, gamma sol_lv = solve_ivp(lotka_volterra, [0, 40], [40.0, 9.0], t_eval=np.linspace(0, 40, 2000), args=params, rtol=1e-10, atol=1e-12) The system is genuinely nonlinear (the \\(PQ\\) term couples the two) and has no closed-form solution. But it has a conserved quantity: $$V = \\delta P - \\gamma\\ln P + \\beta Q - \\alpha\\ln Q$$ which serves as a quality test for the integrator, just as mass conservation will in the kinetics example, further on:\nimport numpy as np from scipy.integrate import solve_ivp def lotka_volterra(t, y, alpha, beta, delta, gamma): prey, predator = y return [alpha * prey - beta * prey * predator, delta * prey * predator - gamma * predator] params = (1.0, 0.02, 0.01, 0.5) # alpha, beta, delta, gamma sol_lv = solve_ivp(lotka_volterra, [0, 40], [40.0, 9.0], t_eval=np.linspace(0, 40, 2000), args=params, rtol=1e-10, atol=1e-12) prey, predator = sol_lv.y V = 0.01 * prey - 0.5 * np.log(prey) + 0.02 * predator - 1.0 * np.log(predator) print(f\u0026#34;relative variation of the invariant: {(V.max() - V.min()) / abs(V.mean()):.2e}\u0026#34;) relative variation of the invariant: 7.30e-10The two panels below show the result in time and in the phase plane:\nLotka-Volterra: the populations oscillate with a lag: the predator only grows after the prey has grown. In the phase portrait, each initial condition generates a closed orbit around the equilibrium at (50, 50). The left panel shows the behavior that gives the model its name. Both populations oscillate, but out of step: the predator only thrives after the prey has thrived, and the predator\u0026rsquo;s collapse comes after the prey\u0026rsquo;s collapse.\nThe right one is the phase portrait, the same tool used on the pendulum: each initial condition generates a closed orbit around the equilibrium at \\((50, 50)\\), which is where both derivatives vanish (\\(P^\\ast = \\gamma/\\delta\\), \\(Q^\\ast = \\alpha/\\beta\\)). A closed orbit means the system returns exactly to the starting point: the oscillations neither damp nor grow, which is a peculiarity of this model, and one of the reasons it is more a caricature than a realistic description of ecosystems.\nExample 3: Charging RC circuit Statement: An uncharged capacitor (\\(V(0)=0\\)) is connected to a \\(5\\) V source through a \\(1\\) k\\(\\Omega\\) resistor and a \\(1\\) mF capacitor. Find the voltage across the capacitor over time.\nSolution: The 1st-order linear ODE \\(RC\\dfrac{dV}{dt} + V = V_0\\) has a known closed-form solution, \\(V(t) = V_0(1 - e^{-t/RC})\\), one more validation with a closed-form solution before moving on to more complex examples:\nimport numpy as np from scipy.integrate import solve_ivp R, C, V0 = 1e3, 1e-3, 5.0 # ohm, F, V tau = R * C def rc_charging(t, V): return [(V0 - V[0]) / tau] t_eval_rc = np.linspace(0, 5 * tau, 200) sol_rc = solve_ivp(rc_charging, [0, 5 * tau], [0.0], t_eval=t_eval_rc, rtol=1e-8, atol=1e-10) V_analytic = V0 * (1 - np.exp(-t_eval_rc / tau)) print(\u0026#34;maximum error:\u0026#34;, np.max(np.abs(sol_rc.y[0] - V_analytic))) maximum error: 9.250205046384963e-09 Example 4: Falling with quadratic air resistance Statement: An object of mass \\(80\\) kg falls from rest, subject to gravity and to an air resistance proportional to the square of the velocity (\\(F_{\\text{drag}} = -kv^2\\), with \\(k=0.25\\) kg/m, a more realistic model than linear drag at higher velocities). Find the velocity as a function of time.\nSolution: The equation of motion is \\(m\\dfrac{dv}{dt} = mg - kv^2\\). The terminal velocity, where \\(dv/dt=0\\), comes out immediately: \\(v_t = \\sqrt{mg/k}\\).\nimport numpy as np from scipy.integrate import solve_ivp m, g_local, k = 80.0, 9.81, 0.25 # kg, m/s^2, kg/m v_terminal = np.sqrt(m * g_local / k) def fall_with_drag(t, v): return [g_local - (k / m) * v[0] ** 2] t_eval_fall = np.linspace(0, 15, 200) sol_fall = solve_ivp(fall_with_drag, [0, 15], [0.0], t_eval=t_eval_fall, rtol=1e-10, atol=1e-12) print(\u0026#34;terminal velocity:\u0026#34;, round(v_terminal, 2), \u0026#34;m/s\u0026#34;) print(\u0026#34;velocity at t=15s:\u0026#34;, round(sol_fall.y[0][-1], 2), \u0026#34;m/s\u0026#34;) terminal velocity: 56.03 m/s velocity at t=15s: 55.45 m/s At 15 s, the object is already at \\(98.9\\%\\) of the terminal velocity, but formally never reaches it, since \\(v_t\\) is an asymptotic limit, not a value reached in finite time.\nAnd this case still has a closed form, which makes it the third validation among these examples. Separating the variables into \\(dv/(1 - v^2/v_t^2) = g\\ dt\\) and integrating, one arrives at a hyperbolic tangent: $$v(t) = v_t \\tanh\\left(\\frac{g t}{v_t}\\right)$$import numpy as np from scipy.integrate import solve_ivp m, g_local, k = 80.0, 9.81, 0.25 # kg, m/s^2, kg/m v_terminal = np.sqrt(m * g_local / k) def fall_with_drag(t, v): return [g_local - (k / m) * v[0] ** 2] t_eval_fall = np.linspace(0, 15, 200) sol_fall = solve_ivp(fall_with_drag, [0, 15], [0.0], t_eval=t_eval_fall, rtol=1e-10, atol=1e-12) v_analytic = v_terminal * np.tanh(g_local * t_eval_fall / v_terminal) print(\u0026#34;maximum error:\u0026#34;, np.max(np.abs(sol_fall.y[0] - v_analytic))) maximum error: 1.00780468414996e-08 It matches to the eighth decimal place, pure tolerance residue. It is a classic result of mechanics, and it is good that it appears here: \u0026ldquo;has quadratic drag\u0026rdquo; is not a synonym for \u0026ldquo;does not close\u0026rdquo;.\nThe article on orbits and trajectories of this series had already solved a problem of the same family (a projectile under air resistance), but as a transcendental equation for the flight time, with linear drag (\\(F_{\\text{drag}} = -kmv\\), 2D, with a launch angle). Here the physics is different (quadratic drag, vertical component only), but the change of approach is the same lesson: without drag simplified enough to isolate a variable in a closed-form equation, solving the ODE directly in time is the way to go.\nExample 5: Chemical kinetics: consecutive reactions Statement: In a reaction \\(A \\to B \\to C\\), with rate constants \\(k_1=0.3\\) s\\(^{-1}\\) and \\(k_2=0.1\\) s\\(^{-1}\\), starting from \\(A_0=1\\) mol/L and no \\(B\\) or \\(C\\), find the concentrations of the three species over time.\nSolution: The system of coupled ODEs is \\(\\frac{dA}{dt}=-k_1A\\), \\(\\frac{dB}{dt}=k_1A-k_2B\\), \\(\\frac{dC}{dt}=k_2B\\), three chained equations, each feeding the next:\nimport numpy as np from scipy.integrate import solve_ivp k1, k2 = 0.3, 0.1 # 1/s A0 = 1.0 def consecutive_reactions(t, y): A, B, C = y return [-k1 * A, k1 * A - k2 * B, k2 * B] t_eval_kin = np.linspace(0, 30, 300) sol_kin = solve_ivp( consecutive_reactions, [0, 30], [A0, 0.0, 0.0], t_eval=t_eval_kin, rtol=1e-8, atol=1e-10 ) A_analytic = A0 * np.exp(-k1 * t_eval_kin) print(\u0026#34;maximum error in A(t):\u0026#34;, np.max(np.abs(sol_kin.y[0] - A_analytic))) print(\u0026#34;mass conservation deviation:\u0026#34;, np.max(np.abs(sol_kin.y.sum(axis=0) - A0))) maximum error in A(t): 4.733793357303284e-10 mass conservation deviation: 4.440892098500626e-16 Two checks: \\(A(t)\\) matches the pure exponential (since \\(A\\) does not depend on \\(B\\) or \\(C\\)), and the sum \\(A+B+C\\) remains equal to \\(A_0\\) the whole time. That is mass conservation, which no line of code imposed explicitly and which only follows from how the three equations were written.\nAnd we can check more. This system, although coupled, has a closed-form solution, a classic result of chemical kinetics: $$B(t) = \\frac{k_1 A_0}{k_2-k_1}\\left(e^{-k_1t}-e^{-k_2t}\\right)$$import numpy as np from scipy.integrate import solve_ivp k1, k2 = 0.3, 0.1 # 1/s A0 = 1.0 def consecutive_reactions(t, y): A, B, C = y return [-k1 * A, k1 * A - k2 * B, k2 * B] t_eval_kin = np.linspace(0, 30, 300) sol_kin = solve_ivp( consecutive_reactions, [0, 30], [A0, 0.0, 0.0], t_eval=t_eval_kin, rtol=1e-8, atol=1e-10 ) B_analytic = A0 * k1 / (k2 - k1) * (np.exp(-k1 * t_eval_kin) - np.exp(-k2 * t_eval_kin)) print(\u0026#34;maximum error in B(t):\u0026#34;, np.max(np.abs(sol_kin.y[1] - B_analytic))) maximum error in B(t): 7.066555118839801e-10 A third check, and the most demanding of the three: the entire curve of the intermediate, with its characteristic peak, reproduced to the ninth decimal place.\nimport matplotlib.pyplot as plt import numpy as np from scipy.integrate import solve_ivp k1, k2 = 0.3, 0.1 # 1/s A0 = 1.0 def consecutive_reactions(t, y): A, B, C = y return [-k1 * A, k1 * A - k2 * B, k2 * B] t_eval_kin = np.linspace(0, 30, 300) sol_kin = solve_ivp( consecutive_reactions, [0, 30], [A0, 0.0, 0.0], t_eval=t_eval_kin, rtol=1e-8, atol=1e-10 ) fig, ax = plt.subplots() for curve, name in zip(sol_kin.y, [\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;]): ax.plot(t_eval_kin, curve, label=name) ax.set_xlabel(\u0026#34;t (s)\u0026#34;) ax.set_ylabel(\u0026#34;concentration (mol/L)\u0026#34;) ax.legend() The following plot shows the three concentrations over time:\nConsecutive reactions A → B → C: the intermediate B has a characteristic peak before being consumed. Where the Closed-Form Formula Really Breaks # If the system closes, why solve it numerically? Because the formula is fragile in two ways, and both show up early.\nFirst: it blows up when \\(k_1 = k_2\\). Notice the \\(k_2 - k_1\\) in the denominator. It is not a physical singularity (two steps with the same rate constant are perfectly reasonable): it is a singularity of the formula. In Python, the result is literally an exception:\nimport numpy as np A0 = 1.0 t = np.linspace(0, 30, 300) k1 = k2 = 0.2 B_analytic = A0 * k1 / (k2 - k1) * (np.exp(-k1 * t) - np.exp(-k2 * t)) ZeroDivisionError: float division by zero The correct solution in this case exists, but it has a different functional form, in which the difference of exponentials disappears and a factor \\(t\\) enters:\n$$ B(t) = A_0 k t e^{-kt} $$To get there one has to redo the integration from scratch, or take the limit \\(k_2 \\to k_1\\) with L\u0026rsquo;Hôpital. solve_ivp, on the other hand, notices no difference at all. It is the same field, the same code, and the answer comes out right:\nimport numpy as np from scipy.integrate import solve_ivp k1 = k2 = 0.2 # the degenerate case A0 = 1.0 def consecutive_reactions(t, y): A, B, C = y return [-k1 * A, k1 * A - k2 * B, k2 * B] t_eval_kin = np.linspace(0, 30, 300) sol_deg = solve_ivp(consecutive_reactions, [0, 30], [A0, 0.0, 0.0], t_eval=t_eval_kin, rtol=1e-8, atol=1e-10) print(\u0026#34;error against A0*k*t*exp(-k*t):\u0026#34;, np.max(np.abs(sol_deg.y[1] - A0 * 0.2 * t_eval_kin * np.exp(-0.2 * t_eval_kin)))) error against A0*k*t*exp(-k*t): 8.172407195417009e-10 Second: it does not survive a slightly more realistic mechanism. It is enough to make the first step reversible (\\(A \\rightleftharpoons B \\to C\\), with a constant \\(k_{-1}\\) going back) for the system to stop being a cascade and become genuinely coupled:\nk1, k_minus1, k2 = 0.3, 0.15, 0.1 def reversible_reactions(t, y): A, B, C = y return [-k1 * A + k_minus1 * B, k1 * A - k_minus1 * B - k2 * B, k2 * B] Now \\(A\\) depends on \\(B\\), which depends on \\(A\\). Solving this by hand means finding the eigenvalues of a \\(3\\times3\\) matrix (the roots of a cubic characteristic polynomial) and assembling the solution as a combination of three exponentials with those exponents. It is possible, it is ugly, and it vanishes completely as soon as a fourth species or a second-order step is added (where a closed-form solution does not even exist). For solve_ivp, the change is three lines.\nThis is the pattern that repeats throughout this series: the closed form does not disappear all at once, it degrades. First it becomes a special case that needs separate handling, then an expression nobody wants to write, then it no longer exists. The numerical method goes through all three stages without changing code, and that is why it is worth having the analytic validation while it exists: it is what gives confidence to use the method when it runs out.\nThe numerical derivatives article estimated the rate of a 1st-order kinetics from measured concentration data, with no model at all. Here the logic is inverted: starting from the model (the constants \\(k_1\\), \\(k_2\\), and the two-step mechanism), the ODE generates the concentration over time, including the characteristic peak of the intermediate \\(B\\), at \\(t = \\ln(k_1/k_2)/(k_1-k_2) \\approx 5.49\\) s, which a pure 1st-order kinetics would never produce.\nPoints of Attention # The initial condition always matters The same pendulum ODE, with different initial conditions (\\(10°\\), \\(60°\\), or \\(170°\\), all starting from rest), gives curves with different periods: 2.01 s, 2.15 s, and 4.89 s. This is a peculiarity of nonlinear equations worth highlighting: in a linear oscillator, the period does not depend on the amplitude (which is what makes a pendulum clock possible, as long as it swings little). As soon as \\(\\sin\\theta\\) enters without approximation, this independence ends.\nsolve_ivp always solves one specific IVP: change the initial condition, and the whole curve changes. This is quite different from the general family that symbolic dsolve returns, with the constant \\(C_1\\) left open: there, \u0026ldquo;the solution\u0026rdquo; is a family; here, it is always one curve.\nTolerance (atol, rtol) silently affects the result Solving the \\(60°\\) pendulum twice, with different tolerances:\nimport numpy as np from scipy.integrate import solve_ivp g = 9.81 # m/s^2 L = 1.0 # m def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] t_span = [0, 6] t_eval = np.linspace(*t_span, 400) theta0 = np.radians(60) sol_tight = solve_ivp(nonlinear_pendulum, t_span, [theta0, 0.0], t_eval=t_eval, rtol=1e-10, atol=1e-12) sol_loose = solve_ivp(nonlinear_pendulum, t_span, [theta0, 0.0], t_eval=t_eval, rtol=1e-2, atol=1e-2) diff = np.max(np.abs(np.degrees(sol_tight.y[0] - sol_loose.y[0]))) print(\u0026#34;maximum divergence:\u0026#34;, round(diff, 3), \u0026#34;degrees\u0026#34;) maximum divergence: 12.628 degrees More than \\(12°\\) of difference between two \u0026ldquo;apparently identical\u0026rdquo; solve_ivp calls: same ODE, same initial condition, only the tolerance changed.\nNotice where the default sits in this comparison: rtol=1e-3 is between the two, ten times tighter than the loose version and ten million times looser than the tight one. In other words, accepting the default without thinking is not a neutral choice: it is a choice, and on this problem it would still leave error visible to the naked eye after a few oscillations.\nThe practical recipe is simple: solve twice, with different tolerances, and compare. If the two curves coincide, the looser tolerance was already enough. If they do not, tighten until they do. It is the equivalent, for ODEs, of the \u0026ldquo;check the residual\u0026rdquo; that the article on roots, at the start of this series, recommended.\nimport matplotlib.pyplot as plt import numpy as np from scipy.integrate import solve_ivp g = 9.81 # m/s^2 L = 1.0 # m def nonlinear_pendulum(t, y): theta, theta_dot = y return [theta_dot, -(g / L) * np.sin(theta)] t_span = [0, 6] t_eval = np.linspace(*t_span, 400) theta0 = np.radians(60) sol_tight = solve_ivp(nonlinear_pendulum, t_span, [theta0, 0.0], t_eval=t_eval, rtol=1e-10, atol=1e-12) sol_loose = solve_ivp(nonlinear_pendulum, t_span, [theta0, 0.0], t_eval=t_eval, rtol=1e-2, atol=1e-2) fig, ax = plt.subplots() ax.plot(t_eval, np.degrees(sol_tight.y[0]), label=\u0026#34;tight tolerance\u0026#34;) ax.plot(t_eval, np.degrees(sol_loose.y[0]), \u0026#34;--\u0026#34;, label=\u0026#34;loose tolerance (rtol=atol=1e-2)\u0026#34;) ax.set_xlabel(\u0026#34;t (s)\u0026#34;) ax.set_ylabel(r\u0026#34;$\\theta$ (degrees)\u0026#34;) ax.legend() The two curves start together and come apart over time, the same kind of progressive divergence already seen between the nonlinear pendulum and its linear approximation, now caused only by numerical tolerance:\nSame ODE, same initial condition: numerical tolerance alone is enough for visibly different curves. Summary Table # Tool / concept Use Note dsolve (SymPy) General symbolic solution, with open constants Only works when a closed form exists solve_ivp(field, t_span, y0) Numerical solution of a specific IVP field(t, y) returns the derivative of each component of y dense_output=True Allows evaluating the solution at any point after solving Uses sol.sol(t) Explicit method Computes \\(y_{n+1}\\) directly, using only known values Simple computation, no equation to solve, but unstable on stiff problems with a large step Implicit method \\(y_{n+1}\\) appears on both sides of the equation More expensive per step (requires solving the equation), but much more stable Order \\(p(q)\\) Embedded pair: advances with the formula of order \\(p\\), estimates the error with the one of order \\(q\\) Basis of the adaptive step size, error estimated without extra evaluations method=\u0026quot;RK45\u0026quot; (default) Explicit Runge-Kutta Fast in most cases; inefficient on stiff problems method=\u0026quot;Radau\u0026quot; / \u0026quot;BDF\u0026quot; Implicit methods Choice for stiff systems method=\u0026quot;LSODA\u0026quot; Detects stiffness automatically Switches method on its own Symptom of stiffness Number of steps/evaluations far above what is expected Compare len(sol.t) or sol.nfev across methods Initial condition Determines which specific curve is obtained Different from the general family of dsolve Tolerance (rtol, atol) Controls precision vs. computational cost Two solutions with different tolerances can diverge visibly Next Steps # This article covered how to solve a differential equation when there is no closed-form solution, but always starting from an already written model, with the equations and parameters known beforehand. A sibling problem remains: what if what we want is not to solve the ODE, but to find the best parameters for a model, or the minimum (or maximum) point of a function with no closed-form formula for its derivative? In the next article, scipy.optimize tackles exactly this problem, and, later in the series, in an article on optimization applications, the very differential equation solved here comes back as part of the model to be optimized.\n","date":"2026-10-21","externalUrl":null,"permalink":"/2026/10/differential-equations-scipy/","section":"Blog","summary":"The classic pendulum period formula only holds for small oscillations, and the exact equation has no closed-form solution. With SciPy’s solve_ivp, we measure the real period and solve stiff systems, Lotka-Volterra, falling with drag, and consecutive reactions.","title":"Differential Equations with SciPy","type":"posts"},{"content":" Ask SymPy to integrate \\(e^{-x^2}\\) and it won\u0026rsquo;t complain, won\u0026rsquo;t hang, won\u0026rsquo;t return an error. It will answer sqrt(pi)*erf(x)/2, and that answer, however much it may look like a dodge, is the most honest thing it could possibly say.\nBecause \\(\\int e^{-x^2}dx\\) has no elementary antiderivative. There is no finite combination of polynomials, exponentials, logarithms, sines and cosines whose derivative is \\(e^{-x^2}\\). And this is not a limitation of SymPy, nor an open problem waiting for someone clever enough: it is a theorem, proved by Joseph Liouville in the 1830s. The antiderivative exists (every continuous function has one); it just can\u0026rsquo;t be written with the alphabet of functions you learn in high school.\nWhat SymPy did was give it a name: erf, the error function. It is a useful answer, but notice what happened to the original question. If someone wants to know the value of \\(\\int_0^1 e^{-x^2}dx\\) (a number, to put in a report), the answer sqrt(pi)*erf(1)/2 merely pushed the problem inside a new symbol. At some point, someone will have to compute it.\nThis article is about that moment.\nWhy Does Numerical Integration Matter? # Chemistry and thermodynamics: enthalpy change (\\(\\int C_p dT\\)), expansion work (\\(\\int P dV\\)), extent of reaction. Most thermodynamic quantities are an integral, and the \\(C_p\\) and \\(P\\) data rarely come in a form you can integrate on paper. Physics and engineering: center of mass, moment of inertia, work, flux through a surface, accumulated charge. These are all integrals, and almost always of functions that exist only as the result of a measurement or of another simulation. Probability and statistics: practically every probability is the area under a density. And the normal density, \\(\\propto e^{-x^2/2}\\), has the same shape as the \\(e^{-x^2}\\) that opens this article (up to a rescaling of \\(x\\)), and so it has no elementary antiderivative: \\(z\\) tables have existed since before computers. Experimental data: integrating a chromatographic peak to get a concentration, a power curve to get energy consumed, a velocity to get a distance. Here there is no function at all to integrate symbolically: there is a table of numbers. Recapping the Essentials of Symbolic Integration # The article on integrals with SymPy showed sympy.integrate solving indefinite and definite integrals exactly. It is worth recalling the essentials, because the numerical approach is defined against them:\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) print(sp.integrate(x**2, x)) # x**3/3 print(sp.integrate(sp.sin(x), (x, 0, sp.pi))) # 2 When the antiderivative exists and SymPy finds it, use SymPy. An exact answer is better than any approximation, and it can still be differentiated, simplified or combined with other expressions. Numerical integration is not a more modern alternative to symbolic integration: it is what you do when the symbolic kind is not available.\nAnd it stops being available in three quite distinct situations, which this article walks through in this order:\nSituation Example Why SymPy can\u0026rsquo;t solve it The antiderivative does not exist in elementary terms \\(\\int e^{-x^2}dx\\) Liouville\u0026rsquo;s theorem: not a software limitation The antiderivative exists but is useless in practice Long integrands, with dozens of terms The result is an expression nobody can read, and evaluating it numerically costs more than integrating directly There is no function at all A table of measurements There is no expression to integrate; there are only numbers The Integral That Won\u0026rsquo;t Close # Let\u0026rsquo;s start with the case where the impossibility is provable. SymPy receives \\(e^{-x^2}\\) and answers:\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) antiderivative = sp.integrate(sp.exp(-(x**2)), x) print(\u0026#34;antiderivative:\u0026#34;, antiderivative) definite = sp.integrate(sp.exp(-(x**2)), (x, 0, 1)) print(\u0026#34;from 0 to 1:\u0026#34;, definite, \u0026#34;=\u0026#34;, float(definite)) antiderivative: sqrt(pi)*erf(x)/2 from 0 to 1: sqrt(pi)*erf(1)/2 = 0.746824132812427 Liouville\u0026rsquo;s Theorem An elementary function has an elementary antiderivative only when it satisfies a rather restrictive algebraic condition. Functions such as \\(e^{-x^2}\\), \\(\\frac{\\sin x}{x}\\), \\(\\frac{1}{\\ln x}\\) and \\(\\sqrt{1 - k^2\\sin^2 x}\\) provably do not satisfy it.\n\u0026ldquo;Elementary,\u0026rdquo; here, is a technical term: these are the functions constructible from polynomials, exponentials and logarithms by a finite number of sums, products, quotients and compositions, which includes the trigonometric functions, since they are exponentials in disguise.\nThe important consequence is philosophical: when SymPy returns erf, it has not failed. It found the right answer, and the right answer is that this antiderivative needs a new name. It is the same move that gave \\(\\sqrt{2}\\) a symbol of its own when it was proved not to be a ratio of integers.\nAnd erf is not an arbitrary function. It is, up to a normalization, exactly the accumulated area under the Gaussian:\n$$ \\mathrm{erf}(x) = \\frac{2}{\\sqrt{\\pi}} \\int_0^x e^{-t^2} dt $$The factor \\(2/\\sqrt{\\pi}\\) is there so that \\(\\mathrm{erf}\\) tends to \\(1\\) as \\(x \\to \\infty\\), which makes it convenient in probability. The two plots below show the relationship between the two functions, the curve and the area it accumulates:\nThe Gaussian and its accumulated area. The area from 0 to 1 is 0.7468; multiplied by 2/√π, it gives exactly erf(1) = 0.8427. Look at the right panel. erf is a perfectly well-behaved function: smooth, monotonic, bounded between \\(-1\\) and \\(1\\). There is nothing pathological about it. What does not exist is a closed formula for it in elementary terms. That is why erf lives in scipy.special, alongside dozens of other functions that were born the same way: as a name for an integral nobody can write any other way.\nAnd the number we care about comes out the numerical way, directly:\nimport numpy as np import sympy as sp from scipy import integrate x = sp.symbols(\u0026#34;x\u0026#34;) definite = sp.integrate(sp.exp(-(x**2)), (x, 0, 1)) value, error = integrate.quad(lambda t: np.exp(-(t**2)), 0, 1) print(f\u0026#34;quad: {value!r}\u0026#34;) print(f\u0026#34;error estimate: {error:.3e}\u0026#34;) print(f\u0026#34;difference from the symbolic value: {abs(value - float(definite)):.2e}\u0026#34;) quad: 0.7468241328124271 error estimate: 8.291e-15 difference from the symbolic value: 1.11e-16 One line, fifteen correct digits.\nquad: Integration and the Estimate of Its Own Error # scipy.integrate.quad is the standard tool for integrating a function of one variable. It implements adaptive quadrature: the QUADPACK library, written in Fortran in the early 1980s, which SciPy\u0026rsquo;s quad uses under the hood.\nAdaptive quadrature \u0026ldquo;Quadrature\u0026rdquo; is the classical name for numerical integration; it comes from the Greek problem of constructing a square with the same area as a given figure.\nAdaptive is the interesting part. Instead of slicing the interval into equal pieces, the method estimates the error in each piece and subdivides only where the error is large. A stretch where the function is almost a straight line is solved with few evaluations; a stretch where it oscillates is subdivided until the error estimate falls below the tolerance.\nIt is the same idea as the adaptive step that solve_ivp uses in differential equations: spend effort where the problem is hard, not where it is easy.\nWhat sets quad apart from adding up rectangles by hand is the second return value: an estimate of its own error. Few numerical methods offer that, and it is what lets you trust the result without knowing the answer beforehand:\nimport numpy as np from scipy import integrate cases = [ (\u0026#34;exp(-x²) on [0, 1]\u0026#34;, lambda t: np.exp(-(t**2)), 0.0, 1.0, 0.746824132812427), (\u0026#34;sin(x) on [0, π]\u0026#34;, np.sin, 0.0, np.pi, 2.0), (\u0026#34;1/x on [1, 10]\u0026#34;, lambda t: 1 / t, 1.0, 10.0, np.log(10)), ] print(f\u0026#34;{\u0026#39;integral\u0026#39;:\u0026gt;22} {\u0026#39;quad\u0026#39;:\u0026gt;14} {\u0026#39;est. error\u0026#39;:\u0026gt;14} {\u0026#39;actual error\u0026#39;:\u0026gt;12}\u0026#34;) for name, f, a, b, exact in cases: v, e = integrate.quad(f, a, b) print(f\u0026#34;{name:\u0026gt;22} {v:\u0026gt;14.10f} {e:\u0026gt;14.2e} {abs(v - exact):\u0026gt;12.2e}\u0026#34;) integral quad est. error actual error exp(-x²) on [0, 1] 0.7468241328 8.29e-15 1.11e-16 sin(x) on [0, π] 2.0000000000 2.22e-14 0.00e+00 1/x on [1, 10] 2.3025850930 1.75e-08 6.22e-15 Compare the last two columns. In every case the error estimate is conservative: it is larger than the actual error, sometimes by several orders of magnitude. That is on purpose: an estimate that underestimated the error would be far worse than one that exaggerates it. Treat the second value as an upper bound, not as a measurement.\nWhere the Trapezoid and Simpson Rules Come From # quad solves the problem, but as a black box. It is worth opening the box, because the two classical integration rules are geometrically simple and explain the error structure of everything that follows.\nBoth start from the same idea: if I don\u0026rsquo;t know how to integrate \\(f\\), I integrate something similar to \\(f\\) that I do know how to integrate. The difference lies in what is used as \u0026ldquo;something similar\u0026rdquo;:\nRule Approximates the function by Error Trapezoid a line joining each pair of neighboring nodes \\(O(h^2)\\) Simpson a parabola through each triple of nodes \\(O(h^4)\\) With \\(h\\) the width of each subinterval. The figure below draws the two constructions over the same function, with only two subintervals so that the difference is visible:\nTwo lines against a parabola, on the same three nodes. The parabola follows the curvature, and the error drops from 0.0155 to 0.00036, 43 times smaller, with the same number of function evaluations. Note the detail that makes all the difference: the two rules use exactly the same three points. The cost is the same: three function evaluations. What changes is what is done with them. And the gain is forty-three-fold.\nWhy the parabola comes for free It might seem that fitting a parabola would be more expensive than joining two points with a line. It is not. Simpson\u0026rsquo;s rule, once the algebra is done, is a weighted average of the values at the nodes:\n$$\\int_{x_0}^{x_2} f(x) dx \\approx \\frac{h}{3}\\left[f(x_0) + 4f(x_1) + f(x_2)\\right]$$The coefficients \\(1, 4, 1\\) are already the result of integrating the parabola. Nobody needs to solve a system to find it at run time: the closed formula was derived once (Kepler already used it in 1615, and Simpson popularized it in the eighteenth century) and holds forever. It is the same economy as the trapezoid rule, whose weights are \\(1, 1\\).\nThe Difference Between \\(O(h^2)\\) and \\(O(h^4)\\) # The exponents in the table above are not decoration. They say exactly how much you gain by spending more points, and the difference between \\(2\\) and \\(4\\) is large:\nimport numpy as np from scipy import integrate def test_func(t): return np.exp(-(t**2)) EXACT_AREA = 0.746824132812427 print(f\u0026#34;{\u0026#39;n\u0026#39;:\u0026gt;6} {\u0026#39;h\u0026#39;:\u0026gt;10} {\u0026#39;trapezoid err\u0026#39;:\u0026gt;15} {\u0026#39;Simpson err\u0026#39;:\u0026gt;14}\u0026#34;) for n in (4, 8, 16, 32, 64, 128, 256): xs = np.linspace(0, 1, n + 1) ys = test_func(xs) trap_err = abs(integrate.trapezoid(ys, xs) - EXACT_AREA) simp_err = abs(integrate.simpson(ys, x=xs) - EXACT_AREA) print(f\u0026#34;{n:\u0026gt;6} {xs[1] - xs[0]:\u0026gt;10.5f} {trap_err:\u0026gt;15.2e} {simp_err:\u0026gt;14.2e}\u0026#34;) n h trapezoid err Simpson err 4 0.25000 3.84e-03 3.12e-05 8 0.12500 9.59e-04 1.99e-06 16 0.06250 2.40e-04 1.25e-07 32 0.03125 5.99e-05 7.79e-09 64 0.01562 1.50e-05 4.87e-10 128 0.00781 3.74e-06 3.05e-11 256 0.00391 9.36e-07 1.90e-12 Look at the ratios between consecutive rows. Each time \\(n\\) doubles (and so \\(h\\) is halved), the trapezoid error drops by 4 (\\(=2^2\\)) and the Simpson error by 16 (\\(=2^4\\)):\nimport numpy as np from scipy import integrate def test_func(t): return np.exp(-(t**2)) EXACT_AREA = 0.746824132812427 trap_errors, simp_errors = [], [] for n in (4, 8, 16, 32, 64, 128, 256): xs = np.linspace(0, 1, n + 1) ys = test_func(xs) trap_errors.append(abs(integrate.trapezoid(ys, xs) - EXACT_AREA)) simp_errors.append(abs(integrate.simpson(ys, x=xs) - EXACT_AREA)) trap_errors, simp_errors = np.array(trap_errors), np.array(simp_errors) print(\u0026#34;ratio between consecutive errors:\u0026#34;) print(\u0026#34; trapezoid (expected 4 = 2²):\u0026#34;, np.round(trap_errors[:-1] / trap_errors[1:], 2)) print(\u0026#34; Simpson (expected 16 = 2⁴):\u0026#34;, np.round(simp_errors[:-1] / simp_errors[1:], 2)) ratio between consecutive errors: trapezoid (expected 4 = 2²): [4.01 4. 4. 4. 4. 4. ] Simpson (expected 16 = 2⁴): [15.72 15.95 15.99 16. 16. 16. ] This is the same reading of the order of error that the article on numerical derivatives made for finite differences, and it is worth checking the pattern on a log-log plot, where a power law becomes a straight line:\nOn log-log axes, the two lines have slopes 2 and 4, exactly the theoretical orders, marked by the gray bands. The distance between them grows: the finer the mesh, the bigger Simpson\u0026rsquo;s advantage. Simpson doesn\u0026rsquo;t always win The table above is clean because the function is smooth and the interval is well chosen. With few nodes, and on an interval that contains an inflection point, Simpson can come out worse than the trapezoid.\nIn the case of \\(e^{-x^2}\\), the inflection is at \\(x = 1/\\sqrt{2} \\approx 0.707\\). Integrating over \\([0, 2]\\) with only two subintervals, the parabola tries to follow a curvature that changes sign midway, and Simpson\u0026rsquo;s error ends up ten times larger than the trapezoid\u0026rsquo;s: \\(0.052\\) against \\(0.005\\).\nThe \\(O(h^4)\\) order is an asymptotic result: it describes what happens when \\(h\\) gets small enough, not a guarantee for any \\(h\\). That is why adaptive quad is the default choice: it subdivides until the asymptotic behavior actually holds, instead of trusting a fixed mesh chosen at random.\nInfinities and Peaks: Where quad Shines and Where It Fails # Infinite Limits # np.inf is accepted as an integration limit, and quad internally makes a change of variable that brings the infinity into a finite interval:\nimport numpy as np from scipy import integrate v_inf, e_inf = integrate.quad(lambda t: np.exp(-(t**2)), -np.inf, np.inf) print(f\u0026#34;∫ e^(-x²) dx over the whole line = {v_inf:.12f}\u0026#34;) print(f\u0026#34; √π = {np.sqrt(np.pi):.12f}\u0026#34;) ∫ e^(-x²) dx over the whole line = 1.772453850906 √π = 1.772453850906 Twelve correct digits on an integral over the entire line. It is worth noting what this means in practice: don\u0026rsquo;t truncate the tail by hand. The temptation to integrate from \\(-10\\) to \\(10\\) \u0026ldquo;because the rest is negligible\u0026rdquo; trades an exact result for an error you did not control. The figure below quantifies that error for the decaying exponential:\nThe tail of e^(-x) thins out fast, but truncating at x = 2 still leaves 13.5% out. The truncation error decays exponentially, and is entirely avoidable by using np.inf. Truncating at \\(x = 1\\) loses \\(37\\%\\) of the integral; at \\(x = 2\\), still \\(13.5\\%\\); only at \\(x = 8\\) does the error drop to \\(0.03\\%\\). These are numbers far too large for a \u0026ldquo;negligible\u0026rdquo; estimated by eye.\nThe Peak That quad Can\u0026rsquo;t See # All this competence has a limit, and it is important enough to deserve a scare:\nquad can return zero with an estimated error of zero Adaptive quadrature decides where to subdivide based on what it has already seen. If the first sampling of the interval does not touch the region where the function is nonzero, there is no large error to detect, and the algorithm concludes, with full confidence, that the integral is zero.\nimport numpy as np from scipy import integrate WIDTH = 1e-4 peak_func = lambda t: np.exp(-((t - 0.7) / WIDTH) ** 2) # extremely narrow peak at 0.7 exact_area = WIDTH * np.sqrt(np.pi) for kwargs, label in ( ({}, \u0026#34;plain quad\u0026#34;), ({\u0026#34;points\u0026#34;: [0.7]}, \u0026#34;points=[0.7]\u0026#34;), ({\u0026#34;points\u0026#34;: [0.7 - 10 * WIDTH, 0.7 + 10 * WIDTH]}, \u0026#34;points around the peak\u0026#34;), ): v, e = integrate.quad(peak_func, 0, 1, **kwargs) actual_error = abs(v - exact_area) / exact_area * 100 print(f\u0026#34;{label:\u0026gt;24}: {v:.6e} est. error {e:.1e} ACTUAL error {actual_error:7.3f}%\u0026#34;) print(f\u0026#34;{\u0026#39;exact\u0026#39;:\u0026gt;24}: {exact_area:.6e}\u0026#34;) plain quad: 0.000000e+00 est. error 0.0e+00 ACTUAL error 100.000% points=[0.7]: 6.524753e-22 est. error 1.3e-21 ACTUAL error 100.000% points around the peak: 1.772454e-04 est. error 1.8e-10 ACTUAL error 0.000% exact: 1.772454e-04 The first line is the nightmare: result \\(0\\), estimated error \\(0\\). Total confidence in an answer that is one hundred percent wrong.\nThe second line shows that the obvious fix is not enough. The points parameter tells quad where to break the interval, but breaking exactly at \\(0.7\\) puts the peak on the edge of the two subintervals, and the Gauss-Kronrod nodes are interior points of each panel: none falls exactly on the edge, and the one closest to it is still too far away to see so narrow a peak. The peak remains invisible.\nThe third line is the correct use: points surrounding the region of interest, not pointing at it. Then the peak falls in the middle of a subinterval and the error goes to zero.\nThe practical moral: know your function before integrating it. A quick plot of the integrand costs seconds and reveals any fine structure that the quadrature might skip. No adaptive algorithm can find what it never sampled.\nA Family of Integrals at Once # When the integrand returns a vector instead of a scalar, quad_vec solves all the components at once, sharing the evaluations:\nimport numpy as np from scipy import integrate alphas = np.array([0.5, 1.0, 2.0, 4.0]) res, err = integrate.quad_vec(lambda t: np.exp(-alphas * t**2), 0, np.inf) for a, v in zip(alphas, res): print(f\u0026#34;a={a:4.1f}: {v:.10f} √(π/a)/2 = {np.sqrt(np.pi / a) / 2:.10f}\u0026#34;) a= 0.5: 1.2533141373 √(π/a)/2 = 1.2533141373 a= 1.0: 0.8862269255 √(π/a)/2 = 0.8862269255 a= 2.0: 0.6266570687 √(π/a)/2 = 0.6266570687 a= 4.0: 0.4431134627 √(π/a)/2 = 0.4431134627 Integrating a Table: There Is No Function to Integrate # The previous cases had at least a function. Now the most common case in the lab: there are only numbers. A chromatograph recorded signal against time; a datalogger recorded power against time; a stopwatch recorded position against time. There is no expression to differentiate or integrate: there is a table.\nHere trapezoid and simpson stop being a didactic curiosity and become the only option, because they work directly on arrays:\nFunction What it does integrate.trapezoid(y, x) Trapezoid rule over the given points integrate.simpson(y, x=x) Simpson\u0026rsquo;s rule over the given points integrate.cumulative_trapezoid(y, x, initial=0) The cumulative integral, returns an array, not a number None of them treats arbitrary spacing as a problem: the x vector can be irregular, which is normal in real data.\nClosing the Circle: Usain Bolt\u0026rsquo;s Race # The article on numerical derivatives took Usain Bolt\u0026rsquo;s passing times in the Berlin final (2009), the position at every \\(10\\) m, and differentiated them to estimate the instantaneous velocity. Now we go the other way: integrating that velocity should reconstruct the position, and the integral over the whole race must give \\(100\\) m.\nLet\u0026rsquo;s see if it does:\nimport numpy as np from scipy import integrate bolt_distance = np.arange(0, 101, 10) # m, splits every 10 m bolt_time = np.array([0, 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.10, 7.92, 8.75, 9.58]) # s bolt_velocity = np.gradient(bolt_distance, bolt_time) # from the previous article trap_dist = integrate.trapezoid(bolt_velocity, bolt_time) simp_dist = integrate.simpson(bolt_velocity, x=bolt_time) print(f\u0026#34;actual distance: 100 m\u0026#34;) print(f\u0026#34; integrating the velocity (trapezoid): {trap_dist:.3f} m\u0026#34;) print(f\u0026#34; integrating the velocity (Simpson): {simp_dist:.3f} m\u0026#34;) actual distance: 100 m integrating the velocity (trapezoid): 102.230 m integrating the velocity (Simpson): 102.055 m It doesn\u0026rsquo;t: \\(2.23\\) m too many, more than two percent, over a course whose two endpoints are known exactly.\nThis is the kind of result worth more than a perfect closure, because it has a locatable cause. Using cumulative_trapezoid to see where the error is born:\nimport numpy as np from scipy import integrate bolt_distance = np.arange(0, 101, 10) # m, splits every 10 m bolt_time = np.array([0, 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.10, 7.92, 8.75, 9.58]) # s bolt_velocity = np.gradient(bolt_distance, bolt_time) # from the previous article cum_dist = integrate.cumulative_trapezoid(bolt_velocity, bolt_time, initial=0) for t, measured, integrated in zip(bolt_time, bolt_distance, cum_dist): print(f\u0026#34;t={t:5.2f}s measured={measured:5.1f} m integrated={integrated:7.3f} m\u0026#34; f\u0026#34; difference={integrated - measured:+6.3f} m\u0026#34;) t= 0.00s measured= 0.0 m integrated= 0.000 m difference=+0.000 m t= 1.89s measured= 10.0 m integrated= 12.983 m difference=+2.983 m t= 2.88s measured= 20.0 m integrated= 22.426 m difference=+2.426 m t= 3.78s measured= 30.0 m integrated= 32.329 m difference=+2.329 m t= 4.64s measured= 40.0 m integrated= 42.312 m difference=+2.312 m t= 5.47s measured= 50.0 m integrated= 52.257 m difference=+2.257 m t= 6.29s measured= 60.0 m integrated= 62.258 m difference=+2.258 m t= 7.10s measured= 70.0 m integrated= 72.198 m difference=+2.198 m t= 7.92s measured= 80.0 m integrated= 82.199 m difference=+2.199 m t= 8.75s measured= 90.0 m integrated= 92.230 m difference=+2.230 m t= 9.58s measured=100.0 m integrated=102.230 m difference=+2.230 m The right-hand column gives away the culprit. The entire error, almost three meters, shows up already in the first segment, between the starting gun and the \\(10\\) m mark. After that it hardly changes: the difference oscillates between \\(2.20\\) and \\(2.43\\) m until the end of the race.\nAnd the reason lies in the velocity estimated at time zero:\nimport numpy as np bolt_distance = np.arange(0, 101, 10) # m, splits every 10 m bolt_time = np.array([0, 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.10, 7.92, 8.75, 9.58]) # s bolt_velocity = np.gradient(bolt_distance, bolt_time) # from the previous article print(\u0026#34;estimated velocity at each mark (m/s):\u0026#34;, np.round(bolt_velocity, 2)) estimated velocity at each mark (m/s): [ 5.29 8.45 10.63 11.38 11.84 12.12 12.27 12.27 12.12 12.05 12.05] \\(5.29\\) m/s at the start of the race. But Bolt starts from rest: the actual velocity at \\(t = 0\\) is exactly zero. At the edge of the array, np.gradient has no point on the left to make a central difference, and falls back on a one-sided, first-order difference over an interval of \\(1.89\\) s. Precisely the stretch in which the athlete accelerates the hardest in the entire race. The estimate returns something close to the average velocity of the stretch, not the initial one, and integrating that average from \\(t = 0\\) inflates the area.\nThe figure shows both sides of the problem:\nThe area under the velocity curve should give 100 m and gives 102.2. The offset between the two curves on the right is born entirely in the first segment and is carried to the end. Differentiating and integrating back does not return the original What just happened holds as a general rule, because it is a common trap in data processing.\nIn calculus, the fundamental theorem guarantees that differentiating and integrating are inverse operations. Numerically, they are not. Each one introduces its own error, and the errors do not cancel: they add up.\nWorse: integration is a cumulative operation. An error made at the first point does not stay at the first point: it is carried, unchanged, through all the rest of the computation. That is exactly what the table showed: the deviation of \\(+2.98\\) m was born in the first segment and never left.\nWhen you need both, prefer measuring the quantity you care about to differentiating or integrating it from another. And if there is no choice, use the boundary conditions you know (here, \\(v(0) = 0\\)) as a check, not as a fix: forcing \\(v(0) = 0\\) in this case takes the result to \\(97.23\\) m, missing in the other direction. The data are too coarse at the start for any pointwise adjustment to solve it.\nChemistry: Two Everyday Integrals # In the two examples below, the integral is the definition of the quantity we want, not an exercise.\nEnthalpy Change: \\(\\int C_p dT\\) # The heat capacity of a gas is not constant: it grows with temperature, and reference tables give \\(C_p\\) as a polynomial in \\(T\\). The heat needed to warm one mole from \\(T_1\\) to \\(T_2\\) at constant pressure is\n$$ \\Delta H = \\int_{T_1}^{T_2} C_p(T) dT $$ Example 1: Heating CO₂ from 25 °C to 727 °C Statement: For gaseous \\(CO_2\\), \\(C_p(T) = 22.243 + 5.977\\times10^{-2}\\ T - 3.499\\times10^{-5}\\ T^2\\) in J/(mol·K), with \\(T\\) in kelvin. Compute \\(\\Delta H\\) when heating from \\(298.15\\) K to \\(1000\\) K.\nSolution:\nfrom scipy import integrate def cp_co2(T): return 22.243 + 5.977e-2 * T - 3.499e-5 * T**2 delta_H, err = integrate.quad(cp_co2, 298.15, 1000.0) print(f\u0026#34;ΔH = {delta_H:.1f} J/mol = {delta_H / 1000:.2f} kJ/mol\u0026#34;) print(f\u0026#34;estimated error: {err:.1e} J/mol\u0026#34;) ΔH = 31485.5 J/mol = 31.49 kJ/mol estimated error: 3.5e-10 J/mol This is a case where the antiderivative exists and is trivial: it is a degree-3 polynomial. It is worth checking against SymPy:\nimport sympy as sp T = sp.symbols(\u0026#34;T\u0026#34;) cp_sym = 22.243 + 5.977e-2 * T - 3.499e-5 * T**2 print(\u0026#34;SymPy:\u0026#34;, float(sp.integrate(cp_sym, (T, 298.15, 1000.0)))) SymPy: 31485.45630238881 Agreement down to the last digit. When both routes are available, using both is the cheapest form of validation there is.\nWhat interests us here is the comparison with the shortcut that every undergraduate course teaches first, which is to assume constant \\(C_p\\), evaluated at the initial temperature:\nfrom scipy import integrate def cp_co2(T): return 22.243 + 5.977e-2 * T - 3.499e-5 * T**2 delta_H, _ = integrate.quad(cp_co2, 298.15, 1000.0) delta_H_constant = cp_co2(298.15) * (1000.0 - 298.15) print(f\u0026#34;assuming constant Cp at {cp_co2(298.15):.2f} J/(mol·K): \u0026#34; f\u0026#34;{delta_H_constant / 1000:.2f} kJ/mol\u0026#34;) print(f\u0026#34;error of the approximation: {(delta_H_constant - delta_H) / delta_H * 100:+.1f}%\u0026#34;) assuming constant Cp at 36.95 J/(mol·K): 25.94 kJ/mol error of the approximation: -17.6% Almost eighteen percent too low. Over a \\(700\\) K range, the \\(C_p\\) of \\(CO_2\\) grows enough that freezing it at the initial value stops being an approximation and becomes a gross error. The integral costs one line; the approximation costs eighteen percent.\nExpansion Work: \\(\\int P dV\\) # The work done by a gas expanding reversibly is the area under the \\(P(V)\\) curve:\n$$ W = \\int_{V_1}^{V_2} P(V) dV $$For an ideal gas, \\(P = RT/V\\), the antiderivative is the logarithm and the result is the classic formula \\(W = RT\\ln(V_2/V_1)\\). But the article on chemistry and engineering equations showed that real gases at high pressures do not follow the ideal law; they follow, much more faithfully, the van der Waals equation:\n$$ P(V) = \\frac{RT}{V - b} - \\frac{a}{V^2} $$And this one, integrated, also has an elementary antiderivative, but nobody needs to find it to get the number:\nExample 2: Isothermal expansion of CO₂, ideal versus real Statement: One mole of \\(CO_2\\) expands isothermally at \\(300\\) K from \\(0.1\\) to \\(2.0\\) L/mol. Compare the work predicted by the ideal model with that predicted by van der Waals (\\(a = 3.640\\) L²·atm/mol², \\(b = 0.04267\\) L/mol).\nSolution:\nimport numpy as np from scipy import integrate R_GAS = 0.082057 # L·atm/(mol·K) A_VDW, B_VDW = 3.640, 0.04267 T_EXP = 300.0 V1, V2 = 0.1, 2.0 def p_ideal(V): return R_GAS * T_EXP / V def p_vdw(V): return R_GAS * T_EXP / (V - B_VDW) - A_VDW / V**2 w_ideal, _ = integrate.quad(p_ideal, V1, V2) w_vdw, err = integrate.quad(p_vdw, V1, V2) print(f\u0026#34;ideal gas: {w_ideal:8.4f} L·atm/mol\u0026#34;) print(f\u0026#34; (analytic: {R_GAS * T_EXP * np.log(V2 / V1):8.4f} = RT ln(V2/V1))\u0026#34;) print(f\u0026#34;van der Waals: {w_vdw:8.4f} L·atm/mol (estimated error {err:.1e})\u0026#34;) print(f\u0026#34;difference: {(w_vdw - w_ideal) / w_ideal * 100:+.1f}%\u0026#34;) ideal gas: 73.7462 L·atm/mol (analytic: 73.7462 = RT ln(V2/V1)) van der Waals: 52.3310 L·atm/mol (estimated error 5.2e-07) difference: -29.0%Twenty-nine percent difference. And it is no mere model detail: at \\(0.1\\) L/mol the \\(CO_2\\) is compressed to the point where the attractive forces between molecules (the \\(-a/V^2\\) term) push the pressure well below what the ideal law predicts. Less pressure, less area, less work.\nThe plot makes the difference literally visible as an area:\nThe area under the curve is the work. The red band between the two curves, 21.4 L·atm/mol, is how much the ideal model would overestimate the expansion of this real gas. Points of Attention # Don\u0026rsquo;t trust a numerical result you haven\u0026rsquo;t checked The three cheapest checks, in order of cost:\nLook at the error estimate. quad returns it for free. An estimated error of the order of the result itself is a red flag. Plot the integrand. Seconds of work, and it reveals narrow peaks, singularities and oscillations that any adaptive method may skip. Check against an independent route. If SymPy closes it, compare. If there is a known limiting case (the ideal gas, here), compare with it. Simpson requires an even number of subintervals Simpson\u0026rsquo;s rule consumes the nodes three at a time, with overlap, which requires an even number of subintervals (that is, an odd number of points). With an odd number of subintervals, scipy.integrate.simpson does not raise an error: it applies a correction on the last interval, which has a lower order of error than the rest. The result remains usable, but the \\(O(h^4)\\) convergence no longer holds. When the order matters, make sure len(x) is odd.\ntrapezoid and simpson don\u0026rsquo;t estimate error Unlike quad, the rules over tabulated data return only the value. There is no way to ask them for an error estimate, because there is no way to refine the mesh: the data are what they are. The practical way to estimate the error in this case is to compare trapezoid with simpson: if the two agree to many digits, the mesh is fine enough for the curvature of the function. That is what happened with Bolt (\\(102.230\\) against \\(102.055\\)): the two agree, and yet both are \\(2\\%\\) away from the right value. Agreement between methods attests that the mesh is sufficient, not that the input data are correct.\nSummary Table # Tool Use Remark sympy.integrate First attempt, always When it closes, the exact answer is better than any approximation scipy.integrate.quad Integral of a function of one variable Returns (value, error_estimate); the estimate is conservative quad(..., a, np.inf) Improper integrals Better than truncating the tail by hand: the truncation error is large and avoidable quad(..., points=[...]) Fine structure in the integrand Surround the critical region, don\u0026rsquo;t point at it scipy.integrate.quad_vec A family of integrals at once Integrand returns a vector; shares evaluations integrate.trapezoid(y, x) Tabulated data Error \\(O(h^2)\\); accepts irregular spacing integrate.simpson(y, x=x) Tabulated data Error \\(O(h^4)\\); requires an even number of subintervals integrate.cumulative_trapezoid Cumulative integral Returns an array; essential to locate where an error is born Concept Remark Liouville\u0026rsquo;s theorem \\(\\int e^{-x^2}dx\\) has no elementary antiderivative: it is a theorem, not a software limitation Special functions erf, Si, Ei are names for integrals that don\u0026rsquo;t close; they live in scipy.special Order of error Doubling the nodes: trapezoid improves \\(4\\times\\), Simpson \\(16\\times\\), but only in the asymptotic regime Adaptive quadrature Subdivides where the error is large; does not find what it never sampled Cumulative error Integration carries forward the error made at the start, without damping it Differentiating × integrating Numerically they are not inverses: the errors add up, they don\u0026rsquo;t cancel Next Steps # The integrals in this article all have the same structure: the integrand is known beforehand. A function that can be evaluated at any point, or a table already measured. Given that, computing the area is a solved problem.\nBut the most interesting case in applied science is not that one. When the rate of change of a quantity depends on the current value of that same quantity (the speed of a reaction depends on how much reactant is still left, the current depends on the charge already accumulated in the capacitor), there is no integrand to tabulate before starting. Each step depends on the previous one.\nThis stops being an integral and becomes a differential equation, the subject of the next article in the series. And the instrument there, solve_ivp, with its adaptive step, is a close relative of the adaptive quad seen here: the same idea of spending effort where the problem is hard, applied to a problem in which the answer is built as you go along.\n","date":"2026-10-14","externalUrl":null,"permalink":"/2026/10/numerical-integration-scipy/","section":"Blog","summary":"The Gaussian function has no elementary antiderivative, and that is a theorem of Liouville. When the antiderivative does not exist, or all you have is a table of measurements, the way forward is numerical: quad, trapezoid and Simpson, from scipy.integrate, with the care each one demands.","title":"Numerical Integration with SciPy","type":"posts"},{"content":"","date":"2026-10-14","externalUrl":null,"permalink":"/tags/numpy/","section":"tags","summary":"","title":"NumPy","type":"tags"},{"content":" A lab technician watches the temperature of a reactor on a datalogger (a device that automatically records measurements over time): a sequence of numbers read every few seconds, with no formula attached. At some point they need to answer a very concrete question: \u0026ldquo;is the temperature rising too fast right now?\u0026rdquo;. Mathematically, that question is a derivative. But there is no \\(f(x)\\) to differentiate. There are only numbers in a table.\nSymPy computes derivatives of any closed-form expression with a single function call, but it has no answer for this problem: there is no \u0026ldquo;symbolic derivative of a table\u0026rdquo;. This is exactly where numerical derivatives come in: techniques that estimate the rate of change of a function from sampled values alone, with or without a formula behind them.\nWhy Do Numerical Derivatives Matter? # Physics and sensor engineering: accelerometers, position sensors and oscilloscopes deliver sequences of readings over time, and instantaneous velocity and acceleration exist only as numerical derivatives of that data. Chemistry and reaction kinetics: the rate of a reaction at a given instant is the derivative of the concentration with respect to time, estimated from bench measurements, not from a formula known a priori. Control and signal processing: a PID controller (proportional-integral-derivative, a classic automatic control architecture) uses the derivative of the system error in real time, computed on sampled and often noisy data. Numerical optimization: when the objective function has no analytic gradient available (or is too expensive to differentiate symbolically), optimizers fall back on numerical approximations of the derivative, a topic that will return in the optimization article of this series. From Symbolic diff to the Numerical Difference # The SymPy derivatives article used the function \\(f(x) = x^3\\) to introduce diff: the exact derivative, \\(f^{\\prime}(x) = 3x^2\\), evaluated at \\(x_0 = 2\\), is \\(12\\). The idea behind any numerical derivative is to directly approximate the definition of the derivative, the limit of a difference quotient, without actually taking the limit, using a small \\(h\\) instead of \\(h \\to 0\\):\n$$ f'(x_0) \\approx \\frac{f(x_0 + h) - f(x_0 - h)}{2h} $$This is the central difference. Applying it with a few decreasing values of \\(h\\) and comparing with the exact value:\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 f_num = sp.lambdify(x, f_sym) df_exact = sp.lambdify(x, sp.diff(f_sym, x)) x0 = 2.0 exact_value = df_exact(x0) def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) for h in [1e-1, 1e-2, 1e-3, 1e-4]: approx = central_difference(f_num, x0, h) error = abs(approx - exact_value) print(f\u0026#34;h={h:.0e} approximation={approx:.10f} error={error:.2e}\u0026#34;) h=1e-01 approximation=12.0100000000 error=1.00e-02 h=1e-02 approximation=12.0001000000 error=1.00e-04 h=1e-03 approximation=12.0000010000 error=1.00e-06 h=1e-04 approximation=12.0000000100 error=1.00e-08 Each time \\(h\\) shrinks by a factor of 10, the error drops by a factor of about 100. That is the signature of a second-order method, \\(O(h^2)\\), which the next section explains.\nThe same idea rebuilds the tangent line from that article, \\(f(x) = \\frac12 x^2\\) at \\(x_0 = 1\\), without using diff even once:\nimport numpy as np def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) def f_tangent(x): return 0.5 * x**2 x0_tangent = 1.0 fp_tangent = central_difference(f_tangent, x0_tangent, h=1e-5) def tangent_line(x): return f_tangent(x0_tangent) + fp_tangent * (x - x0_tangent) print(\u0026#34;numerical f\u0026#39;(1):\u0026#34;, fp_tangent) numerical f\u0026#39;(1): 1.000000000001 Practically \\(1.0\\), the exact value. The following figure shows the parabola and the tangent line built entirely from this numerical derivative:\nimport matplotlib.pyplot as plt import numpy as np def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) def f_tangent(x): return 0.5 * x**2 x0_tangent = 1.0 fp_tangent = central_difference(f_tangent, x0_tangent, h=1e-5) def tangent_line(x): return f_tangent(x0_tangent) + fp_tangent * (x - x0_tangent) xs = np.linspace(-1, 3, 200) fig, ax = plt.subplots() ax.plot(xs, f_tangent(xs), label=r\u0026#34;$f(x) = \\frac{1}{2}x^2$\u0026#34;) ax.plot(xs, tangent_line(xs), \u0026#34;--\u0026#34;, label=\u0026#34;tangent line (numerical derivative)\u0026#34;) ax.plot(x0_tangent, f_tangent(x0_tangent), \u0026#34;o\u0026#34;, color=\u0026#34;black\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.legend() The same tangent line from the SymPy derivatives article, now built without any symbolic derivative. The Finite Difference Formulas # The central difference is not the only option, and it is worth knowing the other two to understand why it is usually the default choice.\nThree finite difference formulas Forward: \\(\\displaystyle f^{\\prime}(x_0) \\approx \\frac{f(x_0+h) - f(x_0)}{h}\\), error \\(O(h)\\) Backward: \\(\\displaystyle f^{\\prime}(x_0) \\approx \\frac{f(x_0) - f(x_0-h)}{h}\\), error \\(O(h)\\) Central: \\(\\displaystyle f^{\\prime}(x_0) \\approx \\frac{f(x_0+h) - f(x_0-h)}{2h}\\), error \\(O(h^2)\\) Forward and backward use only one \u0026ldquo;side\u0026rdquo; of \\(x_0\\); the central uses both, and this symmetry cancels the first-order error term of the Taylor expansion, which is why it converges much faster.\nThat last sentence deserves the five lines of algebra that justify it, because without them \u0026ldquo;\\(O(h^2)\\)\u0026rdquo; is a memorized label. Everything comes from expanding \\(f\\) in a Taylor series on both sides of \\(x_0\\):\n$$ f(x_0+h) = f + hf' + \\frac{h^2}{2}f'' + \\frac{h^3}{6}f''' + \\cdots $$$$ f(x_0-h) = f - hf' + \\frac{h^2}{2}f'' - \\frac{h^3}{6}f''' + \\cdots $$Forward. Solving the first one for \\(f^{\\prime}\\):\n$$ \\frac{f(x_0+h) - f(x_0)}{h} = f' + \\frac{h}{2}f'' + O(h^2) $$A term proportional to \\(h\\) is left over, hence the \\(O(h)\\).\nCentral. Subtracting the second from the first, all the even-power terms cancel, including \\(\\frac{h^2}{2}f^{\\prime\\prime}\\), which was the villain:\n$$ f(x_0+h) - f(x_0-h) = 2hf' + \\frac{h^3}{3}f''' + \\cdots $$$$ \\frac{f(x_0+h) - f(x_0-h)}{2h} = f' + \\frac{h^2}{6}f''' + O(h^4) $$The first error term is now \\(h^2\\), and the next one is \\(h^4\\). The symmetry eliminates half of the Taylor series for free, and that is what makes the central difference look \u0026ldquo;smarter\u0026rdquo;.\nIt is worth seeing this with an \\(h\\) large enough for the lines to separate in the drawing. For \\(f(x)=x^3\\) at \\(x_0=2\\), with \\(h = 0.9\\):\ndef f(x): return x**3 x0, h = 2.0, 0.9 exact_derivative = 3 * x0**2 for name, approx in ( (\u0026#34;forward\u0026#34;, (f(x0 + h) - f(x0)) / h), (\u0026#34;backward\u0026#34;, (f(x0) - f(x0 - h)) / h), (\u0026#34;central\u0026#34;, (f(x0 + h) - f(x0 - h)) / (2 * h)), ): print(f\u0026#34; {name:\u0026gt;11} = {approx:7.4f} error = {abs(approx - exact_derivative):.4f}\u0026#34;) forward = 18.2100 error = 6.2100 backward = 7.4100 error = 4.5900 central = 12.8100 error = 0.8100 With h = 0.9, the one-sided secants miss in opposite directions: one too high, the other too low. The central one is the average of the two, and so it almost reproduces the slope of the exact tangent. The figure shows the mechanism better than the algebra: the forward difference is off by \\(+6.21\\) and the backward by \\(-4.59\\), in opposite directions. The central is essentially the average of the two, and the two errors cancel almost entirely, leaving \\(0.81\\).\nRepeating the same experiment for \\(f(x) = x^3\\) at \\(x_0 = 2\\) for the three formulas:\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 f_num = sp.lambdify(x, f_sym) df_exact = sp.lambdify(x, sp.diff(f_sym, x)) x0 = 2.0 exact_value = df_exact(x0) def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) def forward_difference(f, x0, h): return (f(x0 + h) - f(x0)) / h def backward_difference(f, x0, h): return (f(x0) - f(x0 - h)) / h print(f\u0026#34;{\u0026#39;h\u0026#39;:\u0026gt;10} {\u0026#39;forward\u0026#39;:\u0026gt;14} {\u0026#39;backward\u0026#39;:\u0026gt;14} {\u0026#39;central\u0026#39;:\u0026gt;14}\u0026#34;) for h in [1e-1, 1e-2, 1e-3, 1e-4]: ep = abs(forward_difference(f_num, x0, h) - exact_value) er = abs(backward_difference(f_num, x0, h) - exact_value) ec = abs(central_difference(f_num, x0, h) - exact_value) print(f\u0026#34;{h:\u0026gt;10.0e} {ep:\u0026gt;14.2e} {er:\u0026gt;14.2e} {ec:\u0026gt;14.2e}\u0026#34;) h forward backward central 1e-01 6.10e-01 5.90e-01 1.00e-02 1e-02 6.01e-02 5.99e-02 1.00e-04 1e-03 6.00e-03 6.00e-03 1.00e-06 1e-04 6.00e-04 6.00e-04 1.00e-08 h forward error backward error central error \\(10^{-1}\\) \\(6.10 \\times 10^{-1}\\) \\(5.90 \\times 10^{-1}\\) \\(1.00 \\times 10^{-2}\\) \\(10^{-2}\\) \\(6.01 \\times 10^{-2}\\) \\(5.99 \\times 10^{-2}\\) \\(1.00 \\times 10^{-4}\\) \\(10^{-3}\\) \\(6.00 \\times 10^{-3}\\) \\(6.00 \\times 10^{-3}\\) \\(1.00 \\times 10^{-6}\\) \\(10^{-4}\\) \\(6.00 \\times 10^{-4}\\) \\(6.00 \\times 10^{-4}\\) \\(1.00 \\times 10^{-8}\\) Forward and backward are off by a factor of \\(\\approx 10\\) for each reduction of \\(h\\) by \\(10\\), the signature of \\(O(h)\\). The central is off by a factor of \\(\\approx 100\\), \\(O(h^2)\\), confirming what the theory predicts.\nDifferentiating Tabulated Data: numpy.gradient # Formula in hand, the real problem (tabulated data, with no formula at all) finally fits. numpy.gradient applies the central difference at the interior points of an array (and the forward/backward difference at the edges, where there is no neighbor on both sides) and accepts non-uniformly spaced coordinates.\nExample: Usain Bolt\u0026rsquo;s Velocity from the Splits Statement: The article on Usain Bolt\u0026#39;s record has Bolt\u0026rsquo;s cumulative times at each 10 m mark of the Berlin 2009 final: position every 10 m, non-uniformly spaced time. Estimate the instantaneous velocity at each mark.\nSolution:\nimport numpy as np bolt_distance = np.arange(0, 101, 10) # m, splits every 10 m bolt_time = np.array( [0, 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.10, 7.92, 8.75, 9.58] ) # s bolt_velocity = np.gradient(bolt_distance, bolt_time) print(\u0026#34;velocities (m/s):\u0026#34;, np.round(bolt_velocity, 2)) print(\u0026#34;maximum velocity:\u0026#34;, np.round(bolt_velocity.max(), 2), \u0026#34;m/s\u0026#34;) velocities (m/s): [ 5.29 8.45 10.63 11.38 11.84 12.12 12.27 12.27 12.12 12.05 12.05] maximum velocity: 12.27 m/sThe peak, \\(12.27\\) m/s, happens between 60 and 70 meters, consistent with what is known about the race. In the original article, the velocity came from fitting a cubic spline to the data and differentiating that spline analytically (CubicSpline(...).derivative()), which gave a maximum velocity of \\(12.37\\) m/s. Here, without fitting anything, applying a finite difference directly to the raw points, the result is \\(0.10\\) m/s away (\\(\u0026lt; 1\\%\\)), good news: for a quick estimate, numpy.gradient already delivers a result almost as good as a more elaborate fit.\nThe following graph shows this evolution:\nimport matplotlib.pyplot as plt import numpy as np bolt_distance = np.arange(0, 101, 10) # m, splits every 10 m bolt_time = np.array( [0, 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.10, 7.92, 8.75, 9.58] ) # s bolt_velocity = np.gradient(bolt_distance, bolt_time) fig, ax = plt.subplots() ax.plot(bolt_time, bolt_velocity, \u0026#34;o-\u0026#34;) ax.set_xlabel(\u0026#34;time (s)\u0026#34;) ax.set_ylabel(\u0026#34;velocity (m/s)\u0026#34;) Usain Bolt\u0026rsquo;s instantaneous velocity (Berlin 2009) estimated by finite differences over the 10 m splits. A Modern API: scipy.differentiate # For closed-form functions (not tabulated data), SciPy has an API dedicated to numerical derivatives, with automatic error estimation and stopping criterion, more robust than picking a single \\(h\\) by hand:\nfrom scipy import differentiate x0 = 2.0 res = differentiate.derivative(lambda x: x**3, x0) print(\u0026#34;f\u0026#39;(2):\u0026#34;, res.df, \u0026#34; estimated error:\u0026#34;, res.error, \u0026#34; evaluations:\u0026#34;, res.nfev) f\u0026#39;(2): 12.0 estimated error: 1.4210854715202004e-14 evaluations: 11 scipy.misc.derivative was removed Older versions of SciPy had scipy.misc.derivative, now completely removed (not merely deprecated). Old code that still imports it breaks with ImportError. The current alternative is scipy.differentiate.derivative (seen above) or a manual central difference like the ones in the previous sections.\nThe Step-Size Trade-off: Truncation × Rounding # Back to the manual central difference: if the error falls with smaller \\(h\\), why not always choose a tiny \\(h\\)? Repeating the experiment with \\(f(x)=x^3\\) at \\(x_0=2\\), but sweeping \\(h\\) over a much wider range, from \\(10^{-16}\\) to \\(10^{-1}\\), reveals the answer:\nimport matplotlib.pyplot as plt import numpy as np import sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 f_num = sp.lambdify(x, f_sym) df_exact = sp.lambdify(x, sp.diff(f_sym, x)) x0 = 2.0 exact_value = df_exact(x0) def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) hs = np.logspace(-16, -1, 200) central_errors = np.array( [abs(central_difference(f_num, x0, h) - exact_value) for h in hs] ) fig, ax = plt.subplots() ax.loglog(hs, central_errors) ax.set_xlabel(\u0026#34;h\u0026#34;) ax.set_ylabel(\u0026#34;absolute error\u0026#34;) Central difference error as a function of h: a well-defined minimum, not a limit as h → 0. import numpy as np import sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 f_num = sp.lambdify(x, f_sym) df_exact = sp.lambdify(x, sp.diff(f_sym, x)) x0 = 2.0 exact_value = df_exact(x0) def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) hs = np.logspace(-16, -1, 200) central_errors = np.array( [abs(central_difference(f_num, x0, h) - exact_value) for h in hs] ) h_opt = hs[np.argmin(central_errors)] print(\u0026#34;approximately optimal h:\u0026#34;, h_opt) print(\u0026#34;minimum error:\u0026#34;, central_errors.min()) approximately optimal h: 8.50448934180267e-06 minimum error: 1.6351364706679306e-11 Central difference: smaller h is not always better For larger \\(h\\), the truncation error dominates: the very approximation of the derivative by a finite difference quotient, which the theory predicts as \\(O(h^2)\\). For \\(h\\) smaller than \\(\\approx 10^{-5}\\) to \\(10^{-6}\\) (in this example, in double-precision floating point), the rounding error dominates: \\(f(x_0+h)\\) and \\(f(x_0-h)\\) become so close that the subtraction loses significant digits. The graph is not a curve decreasing to zero: it is a \u0026ldquo;U\u0026rdquo;, with an optimal \\(h\\) in the middle.\nThe \u0026ldquo;U\u0026rdquo; Is the Sum of Two Lines # The shape is no mystery: it is two simple laws added together, each one dominating one side.\nTruncation. From the Taylor derivation, the error is \\(\\frac{h^2}{6}|f^{\\prime\\prime\\prime}|\\). For \\(f = x^3\\), \\(f^{\\prime\\prime\\prime} = 6\\), so it is exactly \\(h^2\\). Rounding. Each evaluation of \\(f\\) carries a relative error \\(\\epsilon \\approx 2.2\\times10^{-16}\\), hence an absolute error \\(\\epsilon|f|\\). The subtraction in the numerator adds the two, and the division by \\(2h\\) amplifies: the error becomes \\(\\epsilon|f|/h\\). $$ \\text{total error} \\approx \\underbrace{h^2}_{\\text{truncation}} + \\underbrace{\\frac{\\epsilon|f|}{h}}_{\\text{rounding}} $$Differentiating and setting to zero, the minimum is at:\n$$ h_{\\text{opt}} = \\sqrt[3]{\\frac{\\epsilon |f|}{2}} $$import numpy as np import sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_sym = x**3 f_num = sp.lambdify(x, f_sym) df_exact = sp.lambdify(x, sp.diff(f_sym, x)) x0 = 2.0 exact_value = df_exact(x0) def central_difference(f, x0, h): return (f(x0 + h) - f(x0 - h)) / (2 * h) hs = np.logspace(-16, -1, 200) central_errors = np.array( [abs(central_difference(f_num, x0, h) - exact_value) for h in hs] ) machine_eps = np.finfo(float).eps h_theory = (machine_eps * abs(f_num(x0)) / 2) ** (1 / 3) print(f\u0026#34;measured optimal h: {hs[np.argmin(central_errors)]:.3e}\u0026#34;) print(f\u0026#34;predicted optimal h: {h_theory:.3e}\u0026#34;) measured optimal h: 8.504e-06 predicted optimal h: 9.612e-06 Same order of magnitude, with a factor of 1.1 difference, which is good enough, because the rounding model is a rough estimate. And the exponent \\(1/3\\) stops being magical: it comes from balancing \\(h^2\\) against \\(1/h\\).\nThe following graph puts the three curves together, the measured error and the two theoretical terms:\nThe measured error follows the rounding line on the left and the truncation line on the right. The minimum is where they cross, and the vertical line marks the optimal h predicted by the formula. Two things to notice in the figure. First, the measured curve hugs each of the lines in its regime: the two-term model describes well what happens. Second, the left side is jagged and the right side is smooth: the truncation error is deterministic, but the rounding error depends on which bits are left over in each subtraction, and varies erratically. That is why the \u0026ldquo;optimal \\(h\\)\u0026rdquo; is, in practice, a region and not a point; it is not worth hunting for the exact value.\nThe Second Derivative Is Even More Fragile # The same Taylor expansion gives the formula for the second derivative, this time adding the two expansions instead of subtracting:\nSecond-order central difference $$f''(x_0) \\approx \\frac{f(x_0+h) - 2f(x_0) + f(x_0-h)}{h^2}$$ Truncation error \\(O(h^2)\\), like the first derivative. This formula is used in the discretization of partial differential equations.\nBut the \\(h^2\\) in the denominator changes everything on the rounding side: instead of \\(\\epsilon|f|/h\\), the error becomes \\(\\epsilon|f|/h^2\\). The effect is visible in a clean case. For \\(f = x^3\\), the fourth derivative is zero, so the truncation vanishes and what remains is 100% rounding:\nimport sympy as sp x = sp.symbols(\u0026#34;x\u0026#34;) f_num = sp.lambdify(x, x**3) x0 = 2.0 for h in (1e-1, 1e-2, 1e-3, 1e-4, 1e-5): d2 = (f_num(x0 + h) - 2 * f_num(x0) + f_num(x0 - h)) / h**2 print(f\u0026#34;h={h:.0e}: f\u0026#39;\u0026#39;={d2:.8f} error={abs(d2 - 12):.2e}\u0026#34;) h=1e-01: f\u0026#39;\u0026#39;=12.00000000 error=8.88e-15 h=1e-02: f\u0026#39;\u0026#39;=12.00000000 error=2.80e-11 h=1e-03: f\u0026#39;\u0026#39;=12.00000000 error=9.87e-10 h=1e-04: f\u0026#39;\u0026#39;=12.00000028 error=2.82e-07 h=1e-05: f\u0026#39;\u0026#39;=12.00000987 error=9.87e-06 Each division of \\(h\\) by 10 multiplies the error by 100, the signature of \\(1/h^2\\). Notice that here the \u0026ldquo;best\u0026rdquo; \\(h\\) is the largest one tested, the opposite of intuition. For a function with a nonzero fourth derivative there would be a \\(U\\) again, but with the minimum at an \\(h\\) much larger than that of the first derivative.\nWhen the Data Is Noisy # The same U shape appears in another way when the data comes from a real measurement, with noise, and in that case the source of the problem stops being floating-point rounding and becomes the instrument.\nimport numpy as np noise_sigma = 0.01 def noise_experiment(n_points, seed=7): rng = np.random.default_rng(seed + n_points) # seed per call xs = np.linspace(0, 2 * np.pi, n_points) noise = rng.normal(0, noise_sigma, size=xs.shape) noisy_signal = np.sin(xs) + noise numerical_derivative = np.gradient(noisy_signal, xs) error = np.abs(numerical_derivative - np.cos(xs)) return xs, noisy_signal, numerical_derivative, error print(f\u0026#34;{\u0026#39;n points\u0026#39;:\u0026gt;10} {\u0026#39;h\u0026#39;:\u0026gt;10} {\u0026#39;mean error\u0026#39;:\u0026gt;12} {\u0026#39;max error\u0026#39;:\u0026gt;12}\u0026#34;) for n in [20, 50, 200, 1000]: xs, _, _, error = noise_experiment(n) h = xs[1] - xs[0] print(f\u0026#34;{n:\u0026gt;10d} {h:\u0026gt;10.4f} {error.mean():\u0026gt;12.4f} {error.max():\u0026gt;12.4f}\u0026#34;) n points h mean error max error 20 0.3307 0.0239 0.0618 50 0.1282 0.0412 0.1436 200 0.0316 0.1759 0.7145 1000 0.0063 0.8771 4.2044 The signal is \\(\\sin(x)\\) with a small Gaussian noise, \\(\\sigma = 0.01\\), almost imperceptible to the naked eye. Even so, refining the grid makes the derivative worse: with \\(1000\\) points, the mean error (\\(0.88\\)) is already of the same order of magnitude as the amplitude of the cosine we want to recover, which is \\(1\\). In other words: the \u0026ldquo;signal\u0026rdquo; and the \u0026ldquo;error\u0026rdquo; end up with comparable sizes, and the derivative stops meaning much.\nThe reason is that the noise does not change in amplitude when \\(h\\) shrinks, but the real variation of the signal between two neighboring points does; so, as the points get closer, the noise comes to dominate. And this can be predicted before running. In the central difference, the noise of the two samples enters divided by \\(2h\\):\n$$ \\sigma_{\\text{derivative}} \\approx \\frac{\\sqrt{2}\\sigma}{2h} = \\frac{\\sigma}{\\sqrt{2} h} $$With \\(1000\\) points in \\([0, 2\\pi]\\), we have \\(h \\approx 0.0063\\) and \\(\\sigma = 0.01\\), which gives \\(\\sigma_{\\text{derivative}} \\approx 1.12\\). The mean absolute error of a Gaussian is \\(\\sqrt{2/\\pi} \\approx 0.80\\) times its standard deviation, that is, \\(\\approx 0.90\\), matching the \\(0.88\\) measured. Notice the \\(h\\) in the denominator: the derivative error grows when the grid is refined, exactly the opposite of what happens with the truncation error.\nimport matplotlib.pyplot as plt import numpy as np noise_sigma = 0.01 def noise_experiment(n_points, seed=7): rng = np.random.default_rng(seed + n_points) # seed per call xs = np.linspace(0, 2 * np.pi, n_points) noise = rng.normal(0, noise_sigma, size=xs.shape) noisy_signal = np.sin(xs) + noise numerical_derivative = np.gradient(noisy_signal, xs) error = np.abs(numerical_derivative - np.cos(xs)) return xs, noisy_signal, numerical_derivative, error xs_fine, signal_fine, deriv_fine, _ = noise_experiment(200) fig, (ax1, ax2) = plt.subplots(2, 1, sharex=True, figsize=(8, 6)) ax1.plot(xs_fine, signal_fine, \u0026#34;o\u0026#34;, color=\u0026#34;C0\u0026#34;, markersize=3, alpha=0.6, label=\u0026#34;measured signal (noisy)\u0026#34;, zorder=1) ax1.plot(xs_fine, np.sin(xs_fine), color=\u0026#34;C1\u0026#34;, linewidth=2, label=r\u0026#34;true $\\sin(x)$\u0026#34;, zorder=2) ax1.legend() ax2.plot(xs_fine, deriv_fine, color=\u0026#34;C0\u0026#34;, label=\u0026#34;numerical derivative\u0026#34;) ax2.plot(xs_fine, np.cos(xs_fine), color=\u0026#34;C1\u0026#34;, linewidth=2, label=r\u0026#34;true $\\cos(x)$\u0026#34;) ax2.set_xlabel(\u0026#34;x\u0026#34;) ax2.legend() The following figure compares the measured signal with the real one (top) and the numerical derivative with the real derivative (bottom):\nDifferentiating amplifies noise: an almost clean signal (top) produces a visibly degraded derivative (bottom). The same U shape, two different causes In the previous section, the \u0026ldquo;U\u0026rdquo; came from floating-point rounding, a problem that goes away if \\(h\\) is larger. Here, it comes from measurement noise, which no \\(h\\) eliminates by itself: refining the grid only concentrates more noise per unit of \\(h\\). In practice, noisy data sometimes calls for smoothing before differentiating, or a deliberately larger \\(h\\), the opposite of the \u0026ldquo;more points is always better\u0026rdquo; instinct.\nApplication: Reaction Rate from Concentration Data # One last application, leaning toward chemistry: a kinetics experiment measures the concentration of a reactant at times chosen by the experimenter (non-uniformly spaced), without assuming beforehand what the rate law of the reaction is.\nExample: Instantaneous Reaction Rate Without Knowing the Kinetics Statement: A reaction is followed through 11 concentration measurements between \\(t=0\\) and \\(t=30\\) min. Estimate the instantaneous reaction rate, \\(-dC/dt\\), at each measured time, without assuming the order of the reaction.\nSolution:\nimport numpy as np kinetics_time = np.array([0, 2, 4, 6, 8, 10, 12, 15, 20, 25, 30]) # min measured_concentration = np.array( [2.02, 1.456, 1.102, 0.807, 0.598, 0.444, 0.31, 0.208, 0.091, 0.08, 0.024] ) # mol/L instantaneous_rate = -np.gradient(measured_concentration, kinetics_time) print(\u0026#34;rate (mol/L/min):\u0026#34;, np.round(instantaneous_rate, 4)) rate (mol/L/min): [0.282 0.2295 0.1622 0.126 0.0908 0.072 0.0538 0.03 0.0128 0.0067 0.0112]With only the measured data and numpy.gradient, without fitting any kinetic model, we already get the rate at each point, a first step of chemical kinetics analysis. Only after having this curve does it make sense to ask whether it is compatible with a 1st-order law, a 2nd-order one, or something else.\nTo check, I need to \u0026ldquo;cheat\u0026rdquo;. Since the concentration data above was simulated, I know the real relationship behind it: \\(C(t) = C_0 e^{-kt}\\), with \\(k=0.15\\) min\\(^{-1}\\) and \\(C_0=2\\) mol/L, something that, with real bench data, I could never know beforehand. I use this real relationship only to generate a continuous reference curve, evaluated on a fine time grid (not only at the 11 measured instants, otherwise I fall into the same problem of connecting a few points with straight lines from the previous section), and compare it with the estimated rate:\nimport matplotlib.pyplot as plt import numpy as np kinetics_time = np.array([0, 2, 4, 6, 8, 10, 12, 15, 20, 25, 30]) # min measured_concentration = np.array( [2.02, 1.456, 1.102, 0.807, 0.598, 0.444, 0.31, 0.208, 0.091, 0.08, 0.024] ) # mol/L instantaneous_rate = -np.gradient(measured_concentration, kinetics_time) time_fine = np.linspace(0, 30, 200) k_true, C0_true = 0.15, 2.0 # known only because the data is synthetic true_rate_continuous = k_true * C0_true * np.exp(-k_true * time_fine) fig, ax = plt.subplots() ax.plot(kinetics_time, instantaneous_rate, \u0026#34;o\u0026#34;, label=\u0026#34;estimated rate (numpy.gradient)\u0026#34;) ax.plot(time_fine, true_rate_continuous, \u0026#34;--\u0026#34;, label=r\u0026#34;$k \\cdot C_0 e^{-kt}$ (check)\u0026#34;) ax.set_xlabel(\u0026#34;t (min)\u0026#34;) ax.set_ylabel(\u0026#34;reaction rate (mol/L/min)\u0026#34;) ax.legend() The rate estimated from the measured data alone is already compatible with the 1st-order rate law used to generate the data. The rate estimated by finite differences, computed without knowing \\(k\\) or \\(C_0\\), follows the real curve with an error of a few percent over most of the measured range, enough to suspect, correctly, a 1st-order kinetics. But the last two points are way off: at \\(t=20\\) min the error is \\(-14\\%\\), and at \\(t=30\\) min it exceeds \\(200\\%\\). It is worth understanding why, because the explanation is a bench lesson.\nA single bad point contaminates its neighbors The temptation is to blame the edge of the series, and it contributes, but it is not the main cause. The culprit is the point measured at \\(t=25\\) min.\nThis data is synthetic, so we can \u0026ldquo;cheat\u0026rdquo; again and compare each measurement with the true value. Almost all of them fall within \\(1\\sigma\\) of the simulated noise. The one at \\(t=25\\) min does not: it is \\(0.080\\) mol/L against \\(0.047\\) real, a deviation of \\(+3.3\\sigma\\), the kind of point that on a real bench would be called a \u0026ldquo;bad measurement\u0026rdquo;.\nNow notice how the derivative at the edge is computed. With no neighbor on the right, numpy.gradient uses a simple backward difference between the last two points: $$\\frac{C(30) - C(25)}{5} = \\frac{0.024 - 0.080}{5} = -0.0112$$ The outlier enters the computation whole. With the correct value in its place, the same formula would give \\(-0.0046\\), an error of \\(39\\%\\) instead of \\(200\\%\\). The larger spacing (5 min versus 2 min) and the edge make the situation worse, but what quintuples the error is the isolated point.\nThe lesson: differentiating amplifies error, and the damage is local. A single point off the curve does not ruin the whole series: it ruins the derivative at its immediate neighbors. That is why, in practice, one inspects the rate curve looking for exactly this pattern: a pair of discrepant points side by side usually points to a bad measurement between them, not a change of mechanism.\nedge_order: more accurate is not always better numpy.gradient accepts an edge_order parameter, which controls the order of the formula used at the edges. The default is 1: the simple two-point difference (backward at the right edge, like the one we just saw). With edge_order=2, it uses a three-point formula, theoretically more accurate.\nIt seems like the obvious fix for the problem above, but it is not:\nimport numpy as np kinetics_time = np.array([0, 2, 4, 6, 8, 10, 12, 15, 20, 25, 30]) # min measured_concentration = np.array( [2.02, 1.456, 1.102, 0.807, 0.598, 0.444, 0.31, 0.208, 0.091, 0.08, 0.024] ) # mol/L for order in (1, 2): rate = -np.gradient(measured_concentration, kinetics_time, edge_order=order) print(f\u0026#34;edge_order={order}: rate at t=30 = {rate[-1]:.4f} (true: 0.0033)\u0026#34;) edge_order=1: rate at t=30 = 0.0112 (true: 0.0033) edge_order=2: rate at t=30 = 0.0157 (true: 0.0033) The error gets worse, from \\(236\\%\\) to \\(371\\%\\). The reason is the same outlier: the order-2 formula uses three points instead of two, and gives the \\(t=25\\) min point an even greater weight.\nIt is a useful reminder of what \u0026ldquo;order of convergence\u0026rdquo; means. A higher-order formula converges faster when the function is smooth: it is a promise about the truncation error, and only about it. If what dominates the error is measurement noise, not truncation, raising the order amplifies the problem instead of solving it. With smooth data, edge_order=2 really helps; with a bad measurement at the edge, it hurts.\nLater in the series, in the differential equations article, this same kinetics shows up again, but modeled directly as a differential equation, including a case, consecutive reactions \\(A \\to B \\to C\\), with no simple closed-form solution, which numpy.gradient alone would not help predict beforehand.\nAnd the technician from the beginning, watching the reactor temperature on the datalogger? Now they have an answer: numpy.gradient on the latest readings tells them, right away, whether the temperature is rising too fast, with no formula needed, only the numbers the device is already recording.\nSummary Table # Tool / concept Use Note Forward difference \\((f(x+h)-f(x))/h\\) Error \\(O(h)\\); uses only one side of \\(x_0\\) Backward difference \\((f(x)-f(x-h))/h\\) Error \\(O(h)\\); uses only one side of \\(x_0\\) Central difference \\((f(x+h)-f(x-h))/(2h)\\) Error \\(O(h^2)\\); default choice numpy.gradient(y, x) Derivative of tabulated data, x may be non-uniform Central difference at interior points, forward/backward at the edges scipy.differentiate.derivative Numerical derivative of a closed-form function, modern API Automatic error estimate and stopping criterion; replaces scipy.misc.derivative (removed) U curve (h vs. error) Trade-off truncation × rounding An optimal \\(h\\) exists; smaller is not always better Derivative of noisy data numpy.gradient on a signal with measurement noise Refining the grid amplifies the noise, same U shape, different cause Next Steps # This article covered how to compute a derivative when there is no closed formula, only data. The inverse operation has exactly the same problem, and even more acutely: there are integrals that provably have no elementary antiderivative, not because of a SymPy limitation, but by theorem. In the next article, scipy.integrate solves these integrals numerically, and the same Usain Bolt data seen here come back the other way around: the velocity we just differentiated is integrated back, to see whether it reconstructs the \\(100\\) m of the race.\n","date":"2026-10-07","externalUrl":null,"permalink":"/2026/10/numerical-derivatives-scipy/","section":"Blog","summary":"Is the reactor temperature rising too fast? The question is a derivative, but all we have are numbers in a table. We will see finite differences, numpy.gradient and scipy.differentiate, and why shrinking the step does not always improve the derivative, because of rounding and noise.","title":"Numerical Derivatives with SciPy","type":"posts"},{"content":" A chemist calibrating a spectrophotometer follows the usual procedure: she prepares solutions of known concentration, measures the absorbance of each one, and writes down the (concentration, absorbance) pairs. In the end, she has a set of points scattered on a plot, not a formula. The Beer-Lambert law promises a linear relationship between concentration and absorbance, but no measured point falls exactly on any line: there is instrument noise, impurities, temperature variation. The practical question is always the same: which line (or curve) best describes these points, and how much can I trust it?\nThis is not a problem of finding a root, nor of manipulating a polynomial we already know. It is the opposite problem: given the points, find the polynomial. SymPy, for all its symbolic power, has no answer for this: there is no such thing as symbolic regression. This is where NumPy comes in.\nWhy Does Curve Fitting Matter? # Scientific instrumentation: every measuring instrument (from spectrophotometers to precision balances) needs a calibration curve, obtained by fitting a model to known points. Materials and structural engineering: mechanical properties such as elastic modulus, hardness, or fatigue strength come from fitting curves to physical test data, almost always noisy. Experimental physics: recovering a quantity (acceleration, decay constant, frequency) from direct measurements is, almost always, a curve-fitting problem. Data science and machine learning: polynomial regression is the simplest example of a principle central to all machine learning: fitting a model to observed data and evaluating how well it generalizes, instead of merely memorizing the points. Fitting a Polynomial to Data # In the previous article we saw numpy.polynomial.Polynomial as an object: construction, roots, derivative, integral, division, always starting from coefficients we already had. Now we go in the opposite direction: start from experimental points and find the polynomial.\nExample 1: Calibrating a spectrophotometer Statement: A chemist measures the absorbance of 8 solutions of known concentration, between 0 and 5 mmol/L. Fit a line (absorbance as a function of concentration) to the measured data.\nSolution:\nimport numpy as np from numpy.polynomial import Polynomial c_calibration = np.linspace(0, 5, 8) # 8 solutions between 0 and 5 mmol/L A_calibration = np.array([0.0012, 0.1377, 0.2847, 0.4233, 0.5478, 0.6844, 0.8224, 0.9511]) p_calibration = Polynomial.fit(c_calibration, A_calibration, deg=1) print(p_calibration) print(\u0026#34;domain:\u0026#34;, p_calibration.domain) 0.481575 + 0.47485·(-1.0 + 0.4x) domain: [0. 5.]Notice that p_calibration does not print coefficients that are recognizable at first sight. By default, Polynomial.fit() rescales x internally to the interval \\([-1, 1]\\), and the .domain attribute keeps the original interval (here, \\([0, 5]\\), the range of the data). This is the most common source of confusion for anyone trying to compare these coefficients with a manual calculation: they are not on the scale of \\(c\\) in mmol/L, they are on the rescaled scale. To get the coefficients in the natural basis, use .convert():\nimport numpy as np from numpy.polynomial import Polynomial c_calibration = np.linspace(0, 5, 8) # 8 solutions between 0 and 5 mmol/L A_calibration = np.array([0.0012, 0.1377, 0.2847, 0.4233, 0.5478, 0.6844, 0.8224, 0.9511]) p_calibration = Polynomial.fit(c_calibration, A_calibration, deg=1) natural_coefs = p_calibration.convert().coef print(\u0026#34;[intercept, slope]:\u0026#34;, natural_coefs) [intercept, slope]: [0.006725 0.18994 ] The fitted slope, \\(0.190\\), is close to the value used to generate the data, \\(0.20\\); the difference comes from measurement noise.\nA naming detail that often goes unnoticed: the Beer-Lambert law is \\(A = \\varepsilon \\ell c\\), so the slope of the line is not the molar absorptivity \\(\\varepsilon\\) alone, but the product \\(\\varepsilon \\ell\\), the absorptivity times the optical path length of the cuvette. Here the two coincide numerically because \\(\\ell = 1\\) cm, the laboratory standard; with a cuvette of another size, they would no longer coincide.\nThe residuals (the difference between each measured point and the value predicted by the line) show how well the fit describes the data:\nimport numpy as np from numpy.polynomial import Polynomial c_calibration = np.linspace(0, 5, 8) # 8 solutions between 0 and 5 mmol/L A_calibration = np.array([0.0012, 0.1377, 0.2847, 0.4233, 0.5478, 0.6844, 0.8224, 0.9511]) p_calibration = Polynomial.fit(c_calibration, A_calibration, deg=1) residuals = A_calibration - p_calibration(c_calibration) print(\u0026#34;max residual:\u0026#34;, np.max(np.abs(residuals))) max residual: 0.009560714285714145 A maximum residual of \\(0.0096\\) on an absorbance scale that goes up to \\(\\approx 1\\) is a reasonable fit for noisy data.\nBut the number alone says little. What a real analyst looks at is the plot of these residuals, the cheapest diagnostic tool there is in curve fitting:\nimport matplotlib.pyplot as plt import numpy as np from numpy.polynomial import Polynomial c_calibration = np.linspace(0, 5, 8) # 8 solutions between 0 and 5 mmol/L A_calibration = np.array([0.0012, 0.1377, 0.2847, 0.4233, 0.5478, 0.6844, 0.8224, 0.9511]) p_calibration = Polynomial.fit(c_calibration, A_calibration, deg=1) residuals = A_calibration - p_calibration(c_calibration) fig, ax = plt.subplots(figsize=(8, 4.2)) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1.2, zorder=1) ax.stem(c_calibration, residuals, basefmt=\u0026#34; \u0026#34;) ax.set_xlabel(\u0026#34;concentration (mmol/L)\u0026#34;) ax.set_ylabel(\u0026#34;residual (measured − fitted)\u0026#34;) print(\u0026#34;signs:\u0026#34;, \u0026#34;\u0026#34;.join(\u0026#34;+\u0026#34; if r \u0026gt; 0 else \u0026#34;-\u0026#34; for r in residuals)) signs: --++--+- The residuals are spread around zero with no visible pattern, a sign that the linear model is adequate within the measured range. How to read a residual plot The question is not \u0026ldquo;are the residuals small?\u0026rdquo;, but \u0026ldquo;do they have structure?\u0026rdquo;.\nNo pattern, alternating in sign at random, as here (--++--+-): the model captured the shape of the data, and what is left is noise. This is what you want to see. A U-shaped or arc-shaped pattern (all negative at the ends and positive in the middle, or the reverse): the model has the wrong shape, because it lacks curvature. It is the classic sign that degree 1 should be degree 2. A fan opening up: the measurement uncertainty grows with \\(x\\), and the fit should be weighted. Here there is no structure, and that is what makes the example interesting: the curvature exists (the true relationship saturates), but within 0 to 5 mmol/L it is smaller than the instrument noise. No statistical diagnostic could detect it with this data, and that is exactly why the extrapolation disaster, in the next section, is invisible from the calibration. The fit has no way of warning about a regime it has never seen.\nPolynomial.fit() has a legacy version, like poly1d Just as numpy.poly1d is the legacy construction of numpy.polynomial.Polynomial (seen in the previous article), numpy.polyfit is the legacy version of Polynomial.fit(): it takes the same data, but returns only an array of coefficients in decreasing order, not a Polynomial object with .domain built in. Prefer Polynomial.fit() in new code, for the same reason as in the previous article.\nWhy Extrapolating Is Dangerous # The chemist calibrated the instrument between 0 and 5 mmol/L. What happens if we use this same line to predict the absorbance of a much more concentrated solution? To answer, I need to \u0026ldquo;cheat\u0026rdquo;: since the data above was simulated, I know the true relationship behind it, something that, with real data, we could never know beforehand. I use this true relationship only to measure the extrapolation error, not for the fit itself:\nimport numpy as np from numpy.polynomial import Polynomial c_calibration = np.linspace(0, 5, 8) # 8 solutions between 0 and 5 mmol/L A_calibration = np.array([0.0012, 0.1377, 0.2847, 0.4233, 0.5478, 0.6844, 0.8224, 0.9511]) p_calibration = Polynomial.fit(c_calibration, A_calibration, deg=1) def true_absorbance(c): return 0.20 * c / (1 + 0.01 * c) # \u0026#34;true\u0026#34; relationship (unknown in practice) for c_outside in [10, 30, 60]: predicted = p_calibration(c_outside) true = true_absorbance(c_outside) error_pct = 100 * (predicted - true) / true print(f\u0026#34;c={c_outside} mmol/L: predicted={predicted:.3f}, true={true:.3f}, error={error_pct:.1f}%\u0026#34;) c=10 mmol/L: predicted=1.906, true=1.818, error=4.8% c=30 mmol/L: predicted=5.705, true=4.615, error=23.6% c=60 mmol/L: predicted=11.403, true=7.500, error=52.0% Outside the calibrated range, the error grows without warning Real spectrophotometers stop responding linearly at high concentrations, for several reasons at once: stray light in the detector, association between analyte molecules, finite bandwidth of the monochromator. The saturation curve used here, \\(A = \\varepsilon c/(1 + \\alpha c)\\), is a generic stand-in for this set of effects, not the exact model of any of them. All that matters for the argument is the shape: the true response saturates, and the fitted line has no way of knowing this, because it only \u0026ldquo;saw\u0026rdquo; data up to 5 mmol/L. At 10 mmol/L (twice the calibrated range), the error is already \\(4.8\\%\\); at 60 mmol/L, it reaches \\(52\\%\\). A fit is a statement about the observed data, not a universal law: it holds only within the measured range.\nThe following plot shows the fitted line (dashed), the true relationship with saturation (solid), and the calibration points, with the calibrated range highlighted:\nOutside the calibrated range (shaded area), the linear fit diverges more and more from the true relationship, which saturates. When High Degree Is a Problem: Runge\u0026rsquo;s Phenomenon # A natural temptation is to think that, if a degree 1 polynomial fits well, a higher-degree one fits even better, after all, it has more free parameters to adapt to the data. To expose why this intuition fails, we will use the classic function that the German mathematician Carl Runge used to demonstrate the problem, in 1901: \\(f(x) = 1/(1+25x^2)\\).\nThe article on Usain Bolt\u0026#39;s record already mentioned this phenomenon in passing, when choosing splines (low-degree polynomials, one per piece) instead of a single high-degree polynomial to interpolate the athlete\u0026rsquo;s position over time. Here, we reproduce the phenomenon on purpose.\nimport numpy as np from numpy.polynomial import Polynomial def runge_function(x): return 1 / (1 + 25 * x**2) n_points = 15 x_nodes = np.linspace(-1, 1, n_points) y_nodes = runge_function(x_nodes) p_interp = Polynomial.fit(x_nodes, y_nodes, deg=n_points - 1) With 15 equally spaced points, the interpolating polynomial has degree 14: it passes exactly through each point. Let us compare it with the true function on a much finer grid:\nimport numpy as np from numpy.polynomial import Polynomial def runge_function(x): return 1 / (1 + 25 * x**2) x_fine = np.linspace(-1, 1, 2000) y_true = runge_function(x_fine) n_points = 15 x_nodes = np.linspace(-1, 1, n_points) p_interp = Polynomial.fit(x_nodes, runge_function(x_nodes), deg=n_points - 1) interp_error = np.abs(p_interp(x_fine) - y_true) worst_idx = np.argmax(interp_error) print(f\u0026#34;largest error of the degree {n_points - 1} interpolation: {interp_error[worst_idx]:.4f}\u0026#34;) largest error of the degree 14 interpolation: 7.1949 A maximum error of \\(7.19\\), for a function that varies only between 0 and 1! The polynomial passes exactly through the 15 points used to build it, but between them, near the edges of the interval, it oscillates and diverges. Now we compare it with a regression of much lower degree, using many more points (but without requiring the polynomial to pass exactly through each one):\nimport numpy as np from numpy.polynomial import Polynomial def runge_function(x): return 1 / (1 + 25 * x**2) x_fine = np.linspace(-1, 1, 2000) y_true = runge_function(x_fine) x_many = np.linspace(-1, 1, 200) y_many = runge_function(x_many) p_regression = Polynomial.fit(x_many, y_many, deg=6) regression_error = np.abs(p_regression(x_fine) - y_true) print(f\u0026#34;largest error of the degree 6 regression: {np.max(regression_error):.4f}\u0026#34;) largest error of the degree 6 regression: 0.2240 A maximum error of \\(0.22\\), more than 30 times smaller, with a lower-degree polynomial.\nThe conclusion seems obvious: high degree is the villain. But that is the wrong conclusion, and the counterexample fits on the same plot.\nThe Culprit Is Where the Nodes Are # Notice that the comparison above changed three things at once: the degree (14 → 6), the method (exact interpolation → least squares), and the number of points (15 → 200). With three variables changing together, we cannot tell which one produced the improvement.\nLet us isolate. Keeping the degree at exactly 14, the method as exact interpolation, and the number of points at exactly 15, changing only where those 15 points are. Instead of equally spaced, we use the Chebyshev nodes.\nChebyshev nodes The \\(n\\) Chebyshev nodes in the interval \\([-1, 1]\\) are $$x_k = \\cos\\left(\\frac{2k+1}{2n}\\pi\\right), \\qquad k = 0, 1, \\ldots, n-1$$ or, on any interval \\([a,b]\\), shifting and rescaling: $$x_k = \\frac{a+b}{2} + \\frac{b-a}{2}\\cos\\left(\\frac{2k+1}{2n}\\pi\\right)$$ They are exactly the roots of the Chebyshev polynomial of degree \\(n\\), the same family that appears later in this article, now from the other side: there as a representation basis, here as sampling positions.\nThe formula becomes much clearer in its geometric construction: spread \\(n\\) points equally spaced by angle over a semicircle and project each one vertically onto the diameter. The cosine in the formula is literally that projection.\nThe effect of this projection is what matters. Near the top of the arc, the curve is almost horizontal: one angular step shifts the projection a lot. Near the ends, the curve is almost vertical: the same angular step barely moves the projection along the axis, and the points pile up there. The following figure shows the construction, with the equally spaced nodes drawn below for comparison:\nChebyshev nodes are the shadow of equally spaced points on a semicircle. With 11 nodes, the distance between neighbors goes from 0.28 at the center to 0.08 at the edge, a 3.5-fold crowding. With \\(11\\) nodes, the distance between neighbors drops from \\(0.28\\) at the center to \\(0.08\\) at the edge: a \\(3.5\\)-fold crowding exactly where equispaced interpolation failed. In code:\nimport numpy as np from numpy.polynomial import Polynomial def runge_function(x): return 1 / (1 + 25 * x**2) x_fine = np.linspace(-1, 1, 2000) y_true = runge_function(x_fine) def chebyshev_nodes(n, a=-1.0, b=1.0): \u0026#34;\u0026#34;\u0026#34;The n Chebyshev nodes in [a, b]: they crowd near the edges.\u0026#34;\u0026#34;\u0026#34; k = np.arange(n) return 0.5 * (a + b) + 0.5 * (b - a) * np.cos((2 * k + 1) * np.pi / (2 * n)) n_points = 15 x_cheb = chebyshev_nodes(n_points) p_cheb_nodes = Polynomial.fit(x_cheb, runge_function(x_cheb), deg=n_points - 1) print(f\u0026#34;same degree 14, Chebyshev nodes: {np.max(np.abs(p_cheb_nodes(x_fine) - y_true)):.4f}\u0026#34;) same degree 14, Chebyshev nodes: 0.0466 From \\(7.19\\) to \\(0.047\\), 150 times smaller, without lowering the degree at all. It is even 5 times better than the degree 6 regression we just praised. And, if we raise the degree, it gets even better:\nimport numpy as np from numpy.polynomial import Polynomial def runge_function(x): return 1 / (1 + 25 * x**2) x_fine = np.linspace(-1, 1, 2000) y_true = runge_function(x_fine) def chebyshev_nodes(n, a=-1.0, b=1.0): \u0026#34;\u0026#34;\u0026#34;The n Chebyshev nodes in [a, b]: they crowd near the edges.\u0026#34;\u0026#34;\u0026#34; k = np.arange(n) return 0.5 * (a + b) + 0.5 * (b - a) * np.cos((2 * k + 1) * np.pi / (2 * n)) for n in (15, 21, 31): xe, xc = np.linspace(-1, 1, n), chebyshev_nodes(n) ee = np.max(np.abs(Polynomial.fit(xe, runge_function(xe), deg=n - 1)(x_fine) - y_true)) ec = np.max(np.abs(Polynomial.fit(xc, runge_function(xc), deg=n - 1)(x_fine) - y_true)) print(f\u0026#34;n={n:3d} (degree {n-1:3d}): equispaced={ee:10.4f} Chebyshev={ec:9.5f}\u0026#34;) n= 15 (degree 14): equispaced= 7.1949 Chebyshev= 0.04660 n= 21 (degree 20): equispaced= 59.8223 Chebyshev= 0.01533 n= 31 (degree 30): equispaced= 2388.2632 Chebyshev= 0.00206 Two columns, two opposite behaviors. With equally spaced nodes, raising the degree is catastrophic: the error explodes from 7 to 2388. With Chebyshev nodes, for this function, raising the degree improves things monotonically, and degree 30 reaches an error of \\(0.002\\). The following plot shows the three cases side by side:\nSame degree 14 in the first two panels: only where the nodes are changes. Runge\u0026rsquo;s phenomenon is a problem of node spacing, not of degree. Runge\u0026rsquo;s phenomenon is about the nodes, not about the degree It is common to hear \u0026ldquo;a high-degree polynomial oscillates\u0026rdquo; as if it were an unavoidable property. Runge\u0026rsquo;s phenomenon is what happens when interpolating at equally spaced points: near the edges of the interval, the basis polynomials become too large, and, for Runge\u0026rsquo;s function, the error grows exponentially with the degree.\nChebyshev nodes, which are the projections of equally spaced points on a semicircle, and therefore pile up at the ends, minimize this effect. That is why they appear in quadrature and spectral methods.\nSo we are left with a lesson in two parts:\nIf you choose where to sample (a simulation, a function you can evaluate at any point), do not be afraid of high degree: just do not sample at equally spaced points. If the points are given (laboratory measurements, almost always equally spaced and noisy), then do prefer a low-degree least-squares polynomial. Not because high degree oscillates by nature, but because exactly interpolating noisy points on a uniform grid combines both problems at once. The following figure closes the argument by sweeping the degree from 4 to 30, with both choices of nodes:\nTwo curves, opposite directions. With equally spaced nodes the error explodes with the degree; with Chebyshev nodes it falls monotonically. The degree was never the villain. Two curves, opposite directions, same function and same degree at each point. The dotted line marks the amplitude of the function itself: above it, the \u0026ldquo;interpolant\u0026rdquo; no longer bears any relation to what it should approximate.\nAbove degree 30, even the computation breaks down If you extend this sweep, you will find a NumPy RankWarning and numbers that stop making sense: at degree 40, the equispaced error appears smaller than at degree 30. This is not an improvement: it is the least-squares fit itself that became ill-conditioned, for the same reason as the Wilkinson polynomial in the previous article. The Vandermonde matrix with powers up to \\(x^{40}\\) has nearly parallel columns, and the solution is no longer reliable. That is why the plot stops at 30.\nAnd When You Cannot Choose the Nodes? # The Chebyshev way out assumes you choose where to sample, which is true in a simulation and false in a laboratory. Experimental data comes on the grid the instrument produced, almost always uniform.\nIn that case the way out is not to change the nodes; it is to change the family of interpolants. Instead of a single high-degree polynomial, several low-degree polynomials (one per piece) glued together with continuity: a spline.\nimport numpy as np from numpy.polynomial import Polynomial from scipy.interpolate import CubicSpline, PchipInterpolator def runge_function(x): return 1 / (1 + 25 * x**2) x_fine = np.linspace(-1, 1, 2000) y_true = runge_function(x_fine) x_nodes = np.linspace(-1, 1, 15) y_nodes = runge_function(x_nodes) p_interp = Polynomial.fit(x_nodes, y_nodes, deg=14) cubic_spline = CubicSpline(x_nodes, y_nodes) # the SAME 15 equispaced nodes pchip_spline = PchipInterpolator(x_nodes, y_nodes) for name, interp in ((\u0026#34;degree 14 polynomial\u0026#34;, p_interp), (\u0026#34;CubicSpline\u0026#34;, cubic_spline), (\u0026#34;PchipInterpolator\u0026#34;, pchip_spline)): print(f\u0026#34;{name:\u0026gt;22}: max error = {np.max(np.abs(interp(x_fine) - y_true)):.4f}\u0026#34;) degree 14 polynomial: max error = 7.1949 CubicSpline: max error = 0.0025 PchipInterpolator: max error = 0.0170 The same 15 points, the same uniform grid, and the error drops from \\(7.19\\) to \\(0.0025\\). That is almost three thousand times better, and even better than Chebyshev nodes of the same size (\\(0.047\\)).\nThe following figure overlays the two interpolations on the true function:\nSame fifteen equally spaced nodes. The cubic spline (dashed) follows the function; the single degree 14 polynomial shoots up at the edges. The reason is simple: each piece of the spline is a degree 3 polynomial, and degree 3 has no room to oscillate. The price is that the spline is not a single formula, it is a table of pieces, and its higher-order derivative is not continuous. PchipInterpolator pays a little more error in exchange for an extra guarantee: it preserves monotonicity, that is, it does not invent valleys and peaks where the data has none. For calibration data, this is usually worth more than the extra digits.\nWhere this has already appeared on this site The article on Usain Bolt\u0026#39;s record used exactly CubicSpline to interpolate the athlete\u0026rsquo;s position between the 10 m marks, and differentiate it to obtain the velocity. Now we can say why: the splits come on a fixed grid every 10 meters, with no option to choose the nodes, and a single degree 10 polynomial over them would produce exactly the disaster in the figure above.\nSo we are left with a lesson in three parts:\nIf you choose where to sample, do not be afraid of high degree: just do not sample at equally spaced points. If the nodes are given and you want to pass through all of them, use splines. If the data is noisy, do not pass through all of it: fit a low-degree polynomial by least squares. Exactly interpolating noisy points is interpolating the noise. It is the third case that dominates the rest of this article, since we are talking about experimental data.\nPhysics: Recovering the Acceleration of Gravity # Example 2: Free fall Statement: An object is dropped from a height of \\(10\\) m. Its position is measured at 9 instants between \\(0\\) and \\(1.2\\) s, with measurement noise. Fit a degree 2 polynomial and recover the acceleration of gravity.\nSolution: The position in free fall follows \\(y(t) = y_0 + v_0 t - \\frac{1}{2}g t^2\\), a degree 2 polynomial in \\(t\\). The coefficient of the quadratic term is \\(-g/2\\):\nimport numpy as np from numpy.polynomial import Polynomial t_data = np.array([0., 0.15, 0.3, 0.45, 0.6, 0.75, 0.9, 1.05, 1.2]) y_data = np.array([10.0, 9.8987, 9.5508, 8.9810, 8.2224, 7.2140, 6.0328, 4.6380, 2.9292]) # meters, with noise p_fall = Polynomial.fit(t_data, y_data, deg=2) fall_coefs = p_fall.convert().coef g_estimated = -2 * fall_coefs[2] print(f\u0026#34;estimated g: {g_estimated:.3f} m/s²\u0026#34;) estimated g: 9.683 m/s² The true value used to generate the data was \\(9.8\\) m/s², an error of \\(-1.2\\%\\), entirely explained by measurement noise.\nThe following plot shows the nine measured points and the fitted parabola passing between them. Notice that the curve does not pass exactly through any point, and that is precisely why it can estimate \\(g\\) better than any isolated pair of measurements could:\nThe coefficient of the quadratic term of the fit recovers the acceleration of gravity with about 1% error. Engineering: Elastic Modulus and the Limits of Fitting # Example 3: Elastic modulus of a steel Statement: A tensile test measures stress as a function of strain for a structural steel specimen. The material has a true elastic modulus \\(E = 200\\) GPa and, in this simplified model, stops being linear at \\(0.2\\%\\) strain. Fit a line to the data and recover \\(E\\): first using only the elastic region, then using all the test data, including points beyond yielding.\nSolution:\nimport numpy as np from numpy.polynomial import Polynomial # strain below yielding (0.2%), stress in MPa, with noise elastic_strain = np.array([0., 0.000286, 0.000571, 0.000857, 0.001143, 0.001429, 0.001714, 0.002]) elastic_stress = np.array([7.3, 44.9, 118.0, 171.7, 233.9, 287.3, 350.1, 400.1]) p_elastic = Polynomial.fit(elastic_strain, elastic_stress, deg=1) E_correct = p_elastic.convert().coef[1] # strain up to 1%, including points beyond yielding full_strain = np.linspace(0, 0.01, 15) full_stress = np.array([-2.1, 145.2, 287.4, 398.9, 400.7, 406.0, 407.4, 404.0, 406.0, 401.6, 417.0, 410.8, 418.5, 416.2, 423.8]) p_full = Polynomial.fit(full_strain, full_stress, deg=1) E_wrong = p_full.convert().coef[1] print(f\u0026#34;E (elastic region only): {E_correct:.0f} MPa\u0026#34;) print(f\u0026#34;E (including plastic region): {E_wrong:.0f} MPa\u0026#34;) E (elastic region only): 201902 MPa E (including plastic region): 26746 MPa Using only the elastic region, the fit is off by \\(1.0\\%\\) (\\(201{,}902\\) MPa against the true \\(200{,}000\\) MPa), an error consistent with the \\(4\\) MPa noise of the measurements, and not with a flaw of the method. Including points beyond yielding, where the relationship stops being linear, distorts the result by \\(-86.6\\%\\): the fitted \u0026ldquo;modulus\u0026rdquo;, \\(27\\) GPa, describes nothing physical at all.\nThe comparison between the two errors is the point: \\(1\\%\\) is the honest uncertainty of measuring \\(E\\) with this data; \\(87\\%\\) is the result of fitting the wrong model. The two are not on the same scale, and no statistical refinement saves the second.\nThe following plot makes the damage visible: the line fitted only to the elastic region follows the initial stretch closely, while the one fitted to all the points is flattened and describes neither the elastic nor the plastic region:\nFitting beyond the yield point (dotted line) distorts the computed elastic modulus. \u0026ldquo;0.2%\u0026rdquo; here is not the standard\u0026rsquo;s 0.2% offset Beware of a numerical coincidence: in real tests, the yield strength of a steel is usually defined by the 0.2% offset method, in which a line is drawn parallel to the elastic stretch, shifted by \\(0.002\\) of strain, and yielding is where it crosses the measured curve. This is a measurement procedure, not a property.\nHere, \\(0.002\\) is something else: it is simply the strain at which the synthetic bilinear model changes slope. The two numbers coincide because both come from the same typical order of magnitude for steel, but they are not the same concept.\nThis is the same lesson as the extrapolation with the Beer-Lambert law, only inverted: there, the problem was using the fit outside the range where the data was measured; here, the problem is including, in the fit itself, data from a physical regime (the plastic region) different from the one the linear model intends to describe. In both cases, the root is the same: a fit is only reliable within the regime it actually models.\nOther Bases: Orthogonal Polynomials # All the fits in this article used the \u0026ldquo;natural\u0026rdquo; basis of powers (\\(1, x, x^2, x^3, \\ldots\\)), which is what Polynomial uses by default. But it is not the only possible basis, and numpy.polynomial includes whole families of ready-made orthogonal polynomials: Legendre, Chebyshev, Hermite, Laguerre.\nOrthogonal polynomials A family of polynomials \\(P_0, P_1, P_2, \\ldots\\) is orthogonal on an interval \\([a, b]\\) when, for any two different polynomials of the family, the integral of their product is zero: \\(\\int_a^b P_m(x) P_n(x) w(x) dx = 0\\) for \\(m \\neq n\\), where \\(w(x)\\) is a weight function specific to each family (for Legendre, \\(w(x) = 1\\), on the interval \\([-1, 1]\\)). It is the same concept as orthogonal vectors in linear algebra, only the \u0026ldquo;inner product\u0026rdquo; between two polynomials is this integral, instead of the usual dot product.\nThe following plot shows the first five polynomials of each family:\nThe Legendre (left) and Chebyshev (right) polynomials of degree 0 to 4: two alternative bases to the natural basis of powers of x. We can check this property numerically, by integrating two Legendre polynomials of different degrees and two of the same degree:\nimport numpy as np from numpy.polynomial import Legendre # np.trapezoid requires NumPy \u0026gt;= 2.0; in earlier versions it was called np.trapz x = np.linspace(-1, 1, 200_000) product_different_degrees = np.trapezoid(Legendre.basis(2)(x) * Legendre.basis(3)(x), x) product_same_degree = np.trapezoid(Legendre.basis(2)(x) * Legendre.basis(2)(x), x) print(f\u0026#34;integral P2*P3 (different degrees): {product_different_degrees:.2e}\u0026#34;) print(f\u0026#34;integral P2*P2 (same degree): {product_same_degree:.4f}\u0026#34;) integral P2*P3 (different degrees): 2.78e-17 integral P2*P2 (same degree): 0.4000 The integral between different degrees is, up to rounding error, zero: the orthogonality of the definition, confirmed numerically. Between polynomials of the same degree, the integral does not vanish, and in this case it even matches the known exact value, \\(2/5\\).\nIn practice, Chebyshev.fit() and Legendre.fit() have exactly the same interface as Polynomial.fit(), and can replace it directly. Redoing the free-fall fit (Example 2) with the Chebyshev basis instead of the natural basis:\nimport numpy as np from numpy.polynomial import Chebyshev, Polynomial t_data = np.array([0., 0.15, 0.3, 0.45, 0.6, 0.75, 0.9, 1.05, 1.2]) y_data = np.array([10.0, 9.8987, 9.5508, 8.9810, 8.2224, 7.2140, 6.0328, 4.6380, 2.9292]) # meters, with noise p_cheb = Chebyshev.fit(t_data, y_data, deg=2) g_via_chebyshev = -2 * p_cheb.convert(kind=Polynomial).coef[2] print(f\u0026#34;estimated g (Chebyshev basis): {g_via_chebyshev:.3f} m/s²\u0026#34;) estimated g (Chebyshev basis): 9.683 m/s² The same result as Example 2, \\(9.683\\) m/s², as it should be, since it is the same polynomial, only described in a different basis (that is why .convert(kind=Polynomial) is necessary: without it, the coefficients are in the Chebyshev basis, not in the natural basis, and do not have the same direct interpretation).\nSolving the Wilkinson Polynomial # The practical advantage of orthogonal bases is not just avoiding an exotic object. It attacks head-on the problem that closed the previous article: the Wilkinson polynomial, \\((x-1)(x-2)\\cdots(x-20)\\), whose roots blow up with a tiny perturbation of \\(2^{-23}\\) in a single coefficient. Rebuilding the same polynomial in the Chebyshev basis, but now using what we have already learned about .domain in this section, mapping the domain to the interval \\([1, 20]\\) of the roots, instead of the default \\([-1, 1]\\):\nimport numpy as np from numpy.polynomial import Chebyshev, Polynomial W_natural = Polynomial.fromroots(np.arange(1, 21)) W_cheb = Chebyshev.fromroots(np.arange(1, 21), domain=[1, 20]) print(f\u0026#34;largest |coefficient|, natural basis: {np.max(np.abs(W_natural.coef)):.2e}\u0026#34;) print(f\u0026#34;largest |coefficient|, Chebyshev basis: {np.max(np.abs(W_cheb.coef)):.2e}\u0026#34;) largest |coefficient|, natural basis: 1.38e+19 largest |coefficient|, Chebyshev basis: 3.12e+15 Why the Chebyshev basis is better conditioned The basis of powers \\(1, x, x^2, x^3, \\ldots\\) has a defect that worsens with the degree: its elements become more and more similar to each other. On \\([1, 20]\\), the functions \\(x^{18}\\), \\(x^{19}\\), and \\(x^{20}\\) are practically proportional: all almost zero near 1 and all astronomical near 20. Such a basis is the analog, in function space, of trying to describe a plane using two nearly parallel vectors: any rounding error in the coordinates produces an enormous shift in the result. It is the same ill-conditioned Vandermonde matrix that appeared in the note about RankWarning, above.\nThe Chebyshev polynomials \\(T_n\\) are built for the opposite: they are orthogonal to each other (with an appropriate weight) and bounded by 1 over the whole domain, for any degree. None of them resembles another, and none dominates the others in magnitude. The coordinates in this basis are, therefore, all of the same order of magnitude, and a small relative error in one of them remains small in the result.\nIt is the same idea behind preferring an orthonormal basis in linear algebra, carried over to polynomials.\nThe coefficients are born almost four orders of magnitude smaller. The test that matters, though, is the sensitivity of the roots, perturbing the coefficients of each basis by the same relative amount:\nimport numpy as np from numpy.polynomial import Chebyshev, Polynomial W_natural = Polynomial.fromroots(np.arange(1, 21)) W_cheb = Chebyshev.fromroots(np.arange(1, 21), domain=[1, 20]) rng = np.random.default_rng(0) rel_perturbation = 1e-10 perturbed_natural_coefs = W_natural.coef * (1 + rel_perturbation * rng.normal(size=21)) natural_error = np.max(np.abs( np.sort(W_natural.roots()) - np.sort_complex(Polynomial(perturbed_natural_coefs).roots()) )) rng = np.random.default_rng(0) # same relative perturbation, same seed perturbed_cheb_coefs = W_cheb.coef * (1 + rel_perturbation * rng.normal(size=21)) W_cheb_pert = Chebyshev(perturbed_cheb_coefs, domain=W_cheb.domain, window=W_cheb.window) cheb_error = np.max(np.abs(np.sort(W_cheb.roots()) - np.sort_complex(W_cheb_pert.roots()))) print(f\u0026#34;max error in the roots, natural basis: {natural_error:.4f}\u0026#34;) print(f\u0026#34;max error in the roots, Chebyshev basis: {cheb_error:.2e}\u0026#34;) max error in the roots, natural basis: 4.9929 max error in the roots, Chebyshev basis: 2.32e-07 Comparing with the previous article requires care, because the two perturbations are of different natures. There, we subtracted \\(2^{-23}\\) from a coefficient worth \\(-210\\): an absolute perturbation. Here, we multiply each coefficient by \\((1 + 10^{-10}\\varepsilon)\\): a relative perturbation. To compare them, we need to put both in the same currency:\n$$ \\frac{2^{-23}}{|-210|} = 5.68 \\times 10^{-10} $$That is, in relative terms, the perturbation there was only 5.7 times larger than the one here, not a thousand times. Even so, the effect is brutal: a relative perturbation of \\(10^{-10}\\), of the order of the rounding error that any float64 carries for free, shifts the roots by almost \\(5\\) units in the natural basis.\nIn the Chebyshev basis, with the domain correctly mapped, the same relative perturbation shifts the roots by only \\(2.3 \\times 10^{-7}\\), an error 21 million times smaller.\n≈ 5\nnatural basis\nshift of the roots with a relative perturbation of 10⁻¹⁰ 2.3 × 10⁻⁷\nChebyshev basis\nshift with the same perturbation, domain mapped 21 million ×\nsmaller\nthe error in the Chebyshev basis Why the comparison is fair (and what it does not say) An attentive reader might object: relatively perturbing the coefficients of the two bases is not perturbing the same function. And that is true: the coefficients in the natural basis reach \\(1.4\\times10^{19}\\), so \\(10^{-10}\\) relative changes the value of the polynomial by about \\(10^9\\) near \\(x=20\\); in the Chebyshev basis, whose coefficients are \\(10^{15}\\) and whose basis functions are bounded by 1 on the domain, the same relative \\(10^{-10}\\) changes the value by about \\(10^5\\).\nThe comparison is still the right one in practice, because it is exactly the error that each representation carries when stored in floating point: float64 always errs a little in relative terms, not absolute ones. But the conclusion must be stated carefully: it is not that the polynomial stops being sensitive, but that the Chebyshev basis does not amplify rounding error the way the power basis does.\nWith this caveat: Wilkinson\u0026rsquo;s ill-conditioning is not a curse of the polynomial, it is a consequence of representing it in a bad basis for the interval where the roots live.\nSummary Table # Tool Use Note Polynomial.fit(x, y, deg) Fits a degree deg polynomial to the data Rescales x internally; use .domain and .convert() numpy.polyfit Legacy version of Polynomial.fit() Returns only coefficients, in decreasing order, avoid in new code Residuals y - p(x) Measures how well the fit describes the observed data Extrapolation Evaluate p(x) outside the range of x used in the fit No guarantee whatsoever, the fit only \u0026ldquo;saw\u0026rdquo; the calibrated range Fit outside the valid regime Include, in the fit itself, data from a physical regime different from the one the model describes Inverse of the extrapolation error: elastic modulus distorted by \\(-86.6\\%\\) when including the plastic region Runge\u0026rsquo;s phenomenon Exact high-degree interpolation at equally spaced points Oscillates near the edges; prefer low-degree regression Legendre.fit(), Chebyshev.fit() Fit with orthogonal bases, same interface as Polynomial.fit() Alternative to \\(1, x, x^2, \\ldots\\); same result, better conditioning at high degree Wilkinson polynomial, solved Chebyshev.fromroots() with .domain mapped to where the roots live Error in the roots drops from \\(4.99\\) to \\(2.3\\times10^{-7}\\), 21 million times smaller Next Steps # This article and the previous one form a pair: first we learned to manipulate an already known polynomial as an object (roots, derivative, integral, division), then we learned to find a polynomial from data that comes with no formula attached. Together they cover the essentials of polynomials in NumPy, but they leave out a neighboring problem: what if what we need to compute is not a polynomial, but the derivative of a function that exists only as discrete data, with no closed form? In the next article, we use finite differences and scipy.differentiate to attack exactly this problem.\n","date":"2026-09-30","externalUrl":null,"permalink":"/2026/09/curve-fitting-numpy/","section":"Blog","summary":"Experimental data rarely falls on a curve. We will see how NumPy fits polynomials to noisy data, what SymPy cannot do, why extrapolating is dangerous, and why Runge’s phenomenon comes from equally spaced nodes, not from high degree.","title":"Curve Fitting with NumPy","type":"posts"},{"content":" In 1959, the English mathematician James H. Wilkinson was investigating why root-finding algorithms, considered correct on paper, behaved badly on the first digital computers. To test a hypothesis, he built a deliberately harmless polynomial, now known as Wilkinson\u0026rsquo;s polynomial:\n$$ (x-1)(x-2)(x-3)\\cdots(x-20) $$Degree 20, twenty integer roots, from 1 to 20, all well separated from one another. There is no more \u0026ldquo;polite\u0026rdquo; polynomial than this one, or so it seemed. Wilkinson expanded the product into its 21 coefficients and fed them to a numerical algorithm.\nAnd those coefficients are monstrous. The constant term is \\(20!\\), a 19-digit integer; the largest of all, the coefficient of \\(x^2\\), exceeds \\(1.38 \\times 10^{19}\\). Keep that number in mind: the largest integer that a float64 represents without any error is \\(2^{53} \\approx 9 \\times 10^{15}\\), about a thousand times smaller. In other words, a good part of these coefficients does not even fit exactly in a float: the damage starts before any algorithm runs.\nThe result left Wilkinson perplexed enough that, decades later, he described that moment as \u0026ldquo;the most traumatic experience in my career as a numerical analyst\u0026rdquo;.\nWhat happened? That is what we will reconstruct in this article, with real code, after learning to handle polynomials as objects in NumPy.\nWhy Do Polynomials as Objects Matter? # Numerical analysis and scientific computing: any method that approximates a function by a polynomial (interpolation, quadrature, differential equation solvers) depends on knowing that high-degree polynomials can amplify small errors dramatically, the very problem Wilkinson discovered. Control engineering: the stability of a linear system (electrical, mechanical) usually boils down to where the roots of a characteristic polynomial are. Knowing that these roots can be sensitive to measurement errors in the coefficients is a practical concern. Computer graphics: Bézier curves and splines, used in most vector editors and game engines, are polynomials handled numerically: evaluated, differentiated, and fitted thousands of times per second. Signal processing: digital filters are described by a ratio of polynomials (the transfer function); the location of the roots of the denominator polynomial (the poles) determines whether the filter is stable. SymPy Polynomials, Revisited in NumPy # In the article Polynomials with SymPy we saw the Poly class, which treats a polynomial as an object (with methods for coefficients, degree, roots, derivative and integral) instead of a mere symbolic expression. NumPy has its equivalent: numpy.polynomial.Polynomial. The essential difference is that here the coefficients are not exact symbols but floating-point numbers (float64), which seems like an irrelevant detail and, as we will see at the end of this article, is not.\nExample 1: Recreating a polynomial from the SymPy article Statement: The SymPy article used \\(Q(x) = x^2 - 25\\) as a recurring example. Build the same polynomial with numpy.polynomial.Polynomial and compare the coefficients with SymPy\u0026rsquo;s.\nSolution: Polynomial takes the coefficients in increasing order of degree, that is, the first element is the constant term, not the one of highest degree (more on this in the next section):\nimport sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) Q_sym = sympy.Poly(x**2 - 25, x) Q_np = Polynomial([-25, 0, 1]) # -25 + 0x + 1x^2 print(\u0026#34;SymPy all_coeffs (descending):\u0026#34;, Q_sym.all_coeffs()) print(\u0026#34;NumPy .coef (ascending):\u0026#34;, Q_np.coef) SymPy all_coeffs (descending): [1, 0, -25] NumPy .coef (ascending): [-25. 0. 1.] Same polynomial, two opposite orders for storing the coefficients.\nTrap No. 1: Coefficient Order # Three different conventions, same polynomial NumPy actually has two ways of representing a polynomial, and neither of them agrees with SymPy:\nRepresentation Coefficient order Example for \\(Q(x)=x^2-25\\) numpy.poly1d (legacy) Descending [1, 0, -25] numpy.polynomial.Polynomial (modern) Ascending [-25, 0, 1] sympy.Poly.all_coeffs() Descending [1, 0, -25] poly1d is NumPy\u0026rsquo;s old API: it still works, but it is considered legacy; the documentation recommends Polynomial for new code. The risk lies in mixing them without noticing, since no convention is the \u0026ldquo;right\u0026rdquo; one: passing the coefficient list the wrong way to the wrong class raises no error at all and silently returns a different polynomial.\nAs a check, the two NumPy constructors arrive at the same roots for the same \\(Q(x)\\), despite the opposite coefficient order:\nimport numpy as np from numpy.polynomial import Polynomial Q_np = Polynomial([-25, 0, 1]) # -25 + 0x + 1x^2 p_legacy = np.poly1d([1, 0, -25]) # descending print(\u0026#34;poly1d.roots:\u0026#34;, p_legacy.roots) print(\u0026#34;Polynomial.roots():\u0026#34;, Q_np.roots()) poly1d.roots: [-5. 5.] Polynomial.roots(): [-5. 5.] Roots, Including Complex Ones # Example 2: Roots of a degree-3 polynomial Statement: The SymPy article also used \\(A(x) = x^3 + 2x + 3\\), which has one real root and two complex ones. Find the roots with Polynomial.roots().\nSolution:\nfrom numpy.polynomial import Polynomial A_np = Polynomial([3, 2, 0, 1]) # 3 + 2x + 0x^2 + 1x^3 print(A_np.roots()) [-1. +0.j 0.5-1.6583124j 0.5+1.6583124j]SymPy needed two separate methods for this (all_roots() for all the roots, real_roots() for the real ones only), because, for SymPy, a real root and a complex root are mathematically distinct objects. Polynomial.roots() makes no such distinction and returns a single numeric array, real when all the roots are real and complex (with zero imaginary part in the real roots) when there is any complex root, because for NumPy a polynomial is, above all, an array of numeric coefficients, the same philosophy as numpy.roots already used in Solving Equations Numerically with SciPy, now applied to polynomials as objects.\nDerivative, Integral and Division # Example 3: Derivative, integral and division of polynomials Statement: Compute the derivative of \\(Q(x)=x^2-25\\) at \\(x=2\\), the integral of \\(Q(x)\\) at \\(x=1\\) (integration constant zero), and the quotient and remainder of dividing \\(A(x)=x^3+2x+3\\) by \\(Q(x)\\).\nSolution:\nfrom numpy.polynomial import Polynomial Q_np = Polynomial([-25, 0, 1]) # -25 + x^2 A_np = Polynomial([3, 2, 0, 1]) # 3 + 2x + x^3 print(\u0026#34;Q\u0026#39;(2):\u0026#34;, Q_np.deriv()(2)) print(\u0026#34;integral of Q up to 1:\u0026#34;, Q_np.integ()(1)) quotient, remainder = divmod(A_np, Q_np) print(\u0026#34;quotient:\u0026#34;, quotient) print(\u0026#34;remainder:\u0026#34;, remainder) Q\u0026#39;(2): 4.0 integral of Q up to 1: -24.666666666666668 quotient: 0.0 + 1.0·x remainder: 3.0 + 27.0·x The same results as in the SymPy article: \\(Q^{\\prime}(2)=4\\), \\(\\int_0^1 Q dx = -\\frac{74}{3} \\approx -24.667\\), and dividing \\(A\\) by \\(Q\\) gives quotient \\(x\\) and remainder \\(27x+3\\). As always in a division, \\(Q \\cdot \\text{quotient} + \\text{remainder} = A\\).\ndivmod here is Python\u0026rsquo;s built-in divmod divmod(A_np, Q_np) is not a NumPy function: it is the language\u0026rsquo;s built-in divmod(). It works because Polynomial implements the special method __divmod__ (along with +, -, *, //, %, **, ==), returning two new Polynomial objects (quotient and remainder). The NumPy documentation on the polynomial convenience classes explicitly lists these overloaded operators.\nWhere Did SymPy\u0026rsquo;s Exactness Go? # In the SymPy introduction article, the integer coefficients of a polynomial were exact: sympy.Integer, with no rounding at all. Here, Q_np.coef is already numpy.float64 from the moment of construction. For a low-degree polynomial like \\(Q\\) or \\(A\\), this changes nothing visible. But as the degree grows, the difference between \u0026ldquo;exact\u0026rdquo; and \u0026ldquo;floating point\u0026rdquo; stops being cosmetic; it is time to go back to the polynomial that opened this article.\nBack to Wilkinson\u0026rsquo;s Polynomial # What Wilkinson discovered has a name, and it is worth defining it before reproducing the experiment:\nIll-conditioning A numerical problem is ill-conditioned when a tiny change in the input data produces a disproportionately large change in the answer, even using a perfectly correct algorithm. It is not a code bug; it is a property of the problem itself.\nTime to reproduce Wilkinson\u0026rsquo;s experiment, step by step.\nStep 1: the exact coefficients, with SymPy # First, we expand \\((x-1)(x-2)\\cdots(x-20)\\) symbolically (the same tool used in the previous articles of the series) to get the 21 coefficients with no rounding error:\nimport sympy x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) wilkinson_poly_sym = sympy.Poly(wilkinson_expr, x) coefs_desc = wilkinson_poly_sym.all_coeffs() # descending, exact integers print(\u0026#34;coefficient of x^19:\u0026#34;, coefs_desc[1]) print(\u0026#34;constant term (20!):\u0026#34;, coefs_desc[-1]) coefficient of x^19: -210 constant term (20!): 2432902008176640000 Notice the size of these numbers: the constant term is already \\(20!\\), a 19-digit integer. All 21 coefficients, at this point, are still exact.\nThe following figure shows the magnitude of all of them, on a logarithmic scale, with the line of the largest integer that a float64 stores without any error:\nimport matplotlib.pyplot as plt import numpy as np import sympy x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers magnitudes = [abs(int(c)) for c in reversed(coefs_desc)] # ascending, still exact degrees = np.arange(21) fig, ax = plt.subplots(figsize=(8, 5)) ax.bar(degrees, [float(m) for m in magnitudes], color=\u0026#34;C0\u0026#34;) ax.axhline(2.0**53, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, linewidth=2, label=r\u0026#34;$2^{53}$: largest exact integer in float64\u0026#34;) ax.set_yscale(\u0026#34;log\u0026#34;) ax.set_xlabel(\u0026#34;degree of the term\u0026#34;) ax.set_ylabel(r\u0026#34;$|$coefficient$|$\u0026#34;) ax.legend() print(f\u0026#34;{sum(m \u0026gt; 2**53 for m in magnitudes)} of the 21 coefficients exceed 2^53\u0026#34;) 10 of the 21 coefficients exceed 2^53 Ten of the twenty-one coefficients exceed 2⁵³, the largest integer that a float64 represents exactly. The damage starts at the conversion, before any algorithm runs. Ten of the twenty-one coefficients are above the line. Each of them already reaches float64 with error, not because of any algorithm, but because the format cannot hold the number.\nStep 2: converting to float64, and error already appears # Now comes the part that any NumPy Polynomial requires: converting these exact integers to float64. That alone is already a perturbation:\nimport sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) With the polynomial built, we compute the roots and compare them with the expected values (the integers from 1 to 20):\nimport numpy as np import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) roots = W.roots() max_error = np.max(np.abs(np.sort(roots.real) - np.arange(1, 21))) print(f\u0026#34;maximum error, with no deliberate perturbation: {max_error:.4f}\u0026#34;) print(\u0026#34;roots computed for the expected values 14 and 15:\u0026#34;) print(np.sort_complex(roots)[13:15]) maximum error, with no deliberate perturbation: 0.5185 roots computed for the expected values 14 and 15: [14.518475-0.20853241j 14.518475+0.20853241j] It is worth rereading that output carefully, because it is more dramatic than it looks: the roots stopped being real. Where two integers should be, \\(14\\) and \\(15\\), there is a pair of complex conjugates, \\(14.518 \\pm 0.209i\\).\nAnd none of this was perturbed on purpose. The only \u0026ldquo;insult\u0026rdquo; to the polynomial was converting its exact integer coefficients to float64, which is unavoidable, since it is what any NumPy Polynomial requires.\nBeware of .roots().real It is tempting to write W.roots().real to \u0026ldquo;clean up\u0026rdquo; the result. But .real silently discards the imaginary part: the conjugate pair above becomes two identical values, \\(14.518\\) and \\(14.518\\), and the reader concludes that the roots merely \u0026ldquo;collided\u0026rdquo;. The most important information, that they left the real axis, vanishes without warning.\nWhen you want only the truly real roots, filter instead of truncating: roots[np.abs(roots.imag) \u0026lt; tol].real.\nTo measure how fragile this is, here is an experiment that perturbs nothing on purpose: building the same polynomial by two different paths. One is the one we just used (expand with SymPy, convert to float64); the other is Polynomial.fromroots, which multiplies the factors directly in floating point.\nimport numpy as np import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) W_via_sympy = W W_via_fromroots = Polynomial.fromroots(np.arange(1, 21)) print(\u0026#34;are the coefficients equal?\u0026#34;, np.allclose(W_via_sympy.coef, W_via_fromroots.coef)) print(\u0026#34;largest relative difference:\u0026#34;, np.max(np.abs(W_via_sympy.coef - W_via_fromroots.coef) / np.maximum(np.abs(W_via_sympy.coef), 1))) print() print(\u0026#34;roots 14/15, via SymPy :\u0026#34;, np.sort_complex(W_via_sympy.roots())[13:15]) print(\u0026#34;roots 14/15, via fromroots:\u0026#34;, np.sort_complex(W_via_fromroots.roots())[13:15]) are the coefficients equal? True largest relative difference: 1.5911824568028567e-16 roots 14/15, via SymPy : [14.518475-0.20853241j 14.518475+0.20853241j] roots 14/15, via fromroots: [14.51819694-0.20459637j 14.51819694+0.20459637j] Read these numbers slowly. The coefficients of the two polynomials differ, at most, by \\(1.6\\times10^{-16}\\) relative, that is, by a single bit in the last digit of the float64. It is the smallest difference that exists between two different numbers in this representation; any smaller and they would be identical.\nAnd the roots differ in the fourth decimal place: the imaginary part goes from \\(0.2085\\) to \\(0.2046\\), almost \\(2\\%\\).\nNeither path is wrong, and no algorithm failed. Simply, in float64, there is no single answer to the question \u0026ldquo;what are the roots of this polynomial\u0026rdquo;: the question is too sensitive for the format in which it was asked. It is this phenomenon that has a name, and it is what we will perturb on purpose next, to measure it.\nStep 3: Wilkinson\u0026rsquo;s classic perturbation # Now, deliberately, we decrease a single coefficient, the one of \\(x^{19}\\), which is exactly \\(-210\\), by just \\(2^{-23} \\approx 1.19\\times10^{-7}\\). This exponent is the machine epsilon of single precision, the 32-bit format with 23 significand bits that we detailed in the article on IEEE 754. It is an error smaller than that of any reasonable experimental measurement.\nimport numpy as np import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) perturbed_coefs = np.array(wilkinson_coefs, dtype=float) perturbed_coefs[19] -= 2**-23 W_perturbed = Polynomial(perturbed_coefs) Step 4: what happened to the roots # With the perturbed polynomial built, we recompute the roots and compare them with the original ones:\nimport numpy as np import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) perturbed_coefs = np.array(wilkinson_coefs, dtype=float) perturbed_coefs[19] -= 2**-23 W_perturbed = Polynomial(perturbed_coefs) perturbed_roots = W_perturbed.roots() max_imag_part = np.max(np.abs(perturbed_roots.imag)) real_roots = perturbed_roots[np.abs(perturbed_roots.imag) \u0026lt; 1e-6].real root_that_was_20 = real_roots[np.argmin(np.abs(real_roots - 20))] print(f\u0026#34;largest imaginary part that appeared: {max_imag_part:.4f}\u0026#34;) print(f\u0026#34;root that was 20 became: {root_that_was_20:.4f}\u0026#34;) largest imaginary part that appeared: 2.8127 root that was 20 became: 20.8469 Ten of the twenty roots stop being real. The root that was exactly 20 rises to \\(\\approx 20.85\\), a deviation of \\(4.2\\%\\) caused by a perturbation of \\(10^{-7}\\) in a single coefficient out of twenty.\nStep 5: visualizing the scattering of the roots # The following plot puts side by side the original roots, all real (circles, aligned on the x axis), and the perturbed roots, scattered across the complex plane (x):\nimport matplotlib.pyplot as plt import numpy as np import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) perturbed_coefs = np.array(wilkinson_coefs, dtype=float) perturbed_coefs[19] -= 2**-23 perturbed_roots = Polynomial(perturbed_coefs).roots() fig, ax = plt.subplots(figsize=(8, 7)) ax.scatter( np.arange(1, 21), np.zeros(20), label=\u0026#34;theoretical roots (integers 1 to 20)\u0026#34;, zorder=3, s=60, ) ax.scatter( perturbed_roots.real, perturbed_roots.imag, marker=\u0026#34;x\u0026#34;, label=\u0026#34;roots after perturbing 1 coefficient by $2^{-23}$\u0026#34;, zorder=3, s=60, ) ax.axhline(0, color=\u0026#34;black\u0026#34;, linewidth=0.8, zorder=1) ax.set_xlabel(\u0026#34;Real part\u0026#34;) ax.set_ylabel(\u0026#34;Imaginary part\u0026#34;) ax.set_title(\u0026#34;Sensitivity of the Wilkinson polynomial roots\u0026#34;, pad=12) ax.legend(loc=\u0026#34;upper center\u0026#34;, bbox_to_anchor=(0.5, -0.12), ncol=1) A perturbation of 2⁻²³ in a single coefficient out of twenty throws ten roots off the real axis. This is what Wilkinson saw in 1959, and the reason his reaction was not an exaggeration.\nWhy This Happens # So far the article has shown that it happens. The why is missing, and it fits in a formula.\nPerturb the coefficient \\(a_k\\) by a small amount \\(\\delta\\). Each root \\(x_j\\) shifts, to first order, by:\n$$ \\frac{\\partial x_j}{\\partial a_k} = -\\frac{x_j^{k}}{p'(x_j)} $$The derivation is short: if \\(p(x_j) = 0\\) and we perturb \\(a_k\\), the new root satisfies \\(p(x_j + \\Delta) + \\delta (x_j+\\Delta)^k = 0\\). Expanding in a Taylor series and keeping the first order, \\(p^{\\prime}(x_j)\\Delta + \\delta x_j^k \\approx 0\\), which gives the ratio above.\nThis fraction explains everything, because its two pieces pull in opposite directions as \\(j\\) grows:\nThe numerator \\(x_j^{19}\\) explodes. For \\(x_1 = 1\\) it is 1; for \\(x_{20} = 20\\) it is \\(5.2 \\times 10^{24}\\). The denominator \\(p^{\\prime}(x_j)\\) plummets in the middle. Since \\(p^{\\prime}(x_j) = \\prod_{i \\neq j}(x_j - x_i)\\), it is the product of the 19 distances from \\(x_j\\) to the other roots. At the ends (\\(j=1\\) or \\(j=20\\)) this product is \\(19! \\approx 10^{17}\\), enormous. In the middle (\\(j \\approx 10\\)), half of the differences are small and the product drops to \\(10^{12}\\). import sympy from numpy.polynomial import Polynomial x = sympy.Symbol(\u0026#34;x\u0026#34;) wilkinson_expr = sympy.prod([x - k for k in range(1, 21)]) coefs_desc = sympy.Poly(wilkinson_expr, x).all_coeffs() # descending, exact integers wilkinson_coefs = [float(c) for c in reversed(coefs_desc)] W = Polynomial(wilkinson_coefs) dW = W.deriv() print(f\u0026#34;{\u0026#39;j\u0026#39;:\u0026gt;3} {\u0026#39;|p_prime(x_j)|\u0026#39;:\u0026gt;15} {\u0026#39;x_j^19\u0026#39;:\u0026gt;12} {\u0026#39;sensitivity\u0026#39;:\u0026gt;15}\u0026#34;) for j in (1, 2, 5, 10, 16, 20): xj = float(j) print(f\u0026#34;{j:\u0026gt;3} {abs(dW(xj)):\u0026gt;15.2e} {xj**19:\u0026gt;12.2e} {abs(xj**19 / dW(xj)):\u0026gt;15.2e}\u0026#34;) j |p_prime(x_j)| x_j^19 sensitivity 1 1.22e+17 1.00e+00 8.22e-18 2 6.40e+15 5.24e+05 8.19e-11 5 3.14e+13 1.91e+13 6.08e-01 10 1.32e+12 1.00e+19 7.60e+06 16 3.14e+13 7.56e+22 2.41e+09 20 1.22e+17 5.24e+24 4.31e+07 The last column varies over 27 orders of magnitude. The root \\(x_1 = 1\\) is essentially immune (\\(8.2 \\times 10^{-18}\\)); the root \\(x_{16}\\) is \\(2.4 \\times 10^{9}\\) times more sensitive than the applied perturbation.\nThe following figure shows that whole column, one bar per root:\nThe sensitivity of each root to the coefficient of x¹⁹, on a log scale. The roots above the dashed line are the ones that the 2⁻²³ perturbation visibly displaces, and they are exactly the ones that break in the experiment. Compare this figure with the result of the experiment: the roots that survived intact are the small ones, and the ones that became complex pairs are those from the middle up. The figure predicted which ones would break, before running roots().\nThis is ill-conditioning, not a bad algorithm The distinction is the moral of the article. numpy.roots computes the eigenvalues of the companion matrix with a norm-wise stable algorithm (backward stable): the result it returns is the exact answer to a problem very close to the one you passed. It does not err: it answers accurately a question slightly different from yours.\nFor well-conditioned problems, that is enough: the exact answer to a nearby problem is a nearby answer. For Wilkinson\u0026rsquo;s polynomial, it is not enough, because \u0026ldquo;close in the coefficients\u0026rdquo; and \u0026ldquo;close in the roots\u0026rdquo; stop being the same thing. Changing the algorithm would not help; the problem is in the question, not in who answers it.\nIt Is Not Just Proximity: Multiplicity Too # It might seem that the problem is having roots that are \u0026ldquo;too close\u0026rdquo;. The opposite extreme shows that it is not:\nimport numpy as np from numpy.polynomial import Polynomial P_mult = Polynomial.fromroots(np.ones(20)) # (x-1)^20 mult_roots = P_mult.roots() print(f\u0026#34;radius of the scatter around 1: {np.max(np.abs(mult_roots - 1)):.4f}\u0026#34;) print(\u0026#34;how many stopped being real:\u0026#34;, int((np.abs(mult_roots.imag) \u0026gt; 1e-8).sum())) radius of the scatter around 1: 0.4352 how many stopped being real: 18 \\((x-1)^{20}\\) has one root, repeated 20 times. roots() returns 20 values scattered on a circle of radius \\(0.44\\) around 1, and 18 of them are not even real. It makes sense by the same formula: at a multiple root, \\(p^{\\prime}(x_j) = 0\\) exactly, and the sensitivity is infinite.\nThe Practical Lesson # High-degree roots seem impossible to tame, but the formula suggests the opposite. It points to the culprit, and the culprit is avoidable:\nDo not expand to find roots The loss happens at the moment the factors \\((x-1)(x-2)\\cdots\\) become a list of coefficients. Before that, the roots were written in plain sight. Practical rules:\nKeep the factored form whenever it exists. Polynomial.fromroots and roots() are inverse operations, but the forward path loses information. Prefer another basis. The power basis \\(1, x, x^2, \\ldots\\) is terrible on intervals far from the origin. The next article shows that the same polynomial, written in the Chebyshev basis with the domain mapped to \\([1, 20]\\), becomes millions of times less sensitive. If the problem comes from a matrix (eigenvalues, system stability), solve the matrix directly instead of building and solving the characteristic polynomial. The path through the polynomial destroys conditioning for free. In other words: Wilkinson\u0026rsquo;s ill-conditioning is not a property of the set of roots \\(\\{1, \\ldots, 20\\}\\). It is a property of representing it as a list of coefficients in the power basis.\nSummary Table # Operation Syntax Note Construction Polynomial([c0, c1, ..., cn]) Ascending order of degree Construction (legacy) numpy.poly1d([cn, ..., c1, c0]) Descending order, avoid in new code Roots .roots() Real array if all roots are real; complex if there is any complex root Derivative .deriv() Returns another Polynomial Integral .integ() Integration constant zero by default Division divmod(A, B) Returns (quotient, remainder) Exact coefficients Does not exist Every coefficient is float64; ill-conditioned at high degree (Wilkinson) Next Steps # This article covered polynomials as objects we already knew, built from coefficients or roots we had in advance. But most real-world data does not come with a polynomial attached: it comes as a cloud of noisy experimental points, and it is up to us to find the polynomial that best describes those points. SymPy does not do that: there is no symbolic regression. In the next article, we use Polynomial.fit() to fit curves to real data from chemistry, physics and engineering, and we see why a higher degree is not always a better fit: Runge\u0026rsquo;s phenomenon, a sort of cousin of Wilkinson\u0026rsquo;s ill-conditioning, but for interpolation instead of roots.\n","date":"2026-09-23","externalUrl":null,"permalink":"/2026/09/polynomials-numpy/","section":"Blog","summary":"Stored in float64, the coefficients of Wilkinson’s polynomial no longer return the roots 1 to 20: two of them become a complex pair. Before getting there, we look at roots, derivatives, integrals and division with numpy.polynomial, and at the coefficient-order trap.","title":"Polynomials with NumPy","type":"posts"},{"content":"","date":"2026-09-16","externalUrl":null,"permalink":"/tags/chemistry/","section":"tags","summary":"","title":"chemistry","type":"tags"},{"content":" In 1873, the Dutch physicist Johannes Diderik van der Waals defended a doctoral thesis that solved a nagging problem: the ideal gas law, \\(PV=nRT\\), simply did not describe real gases near condensation, nor did it explain why, under enough pressure and temperature, a gas becomes a liquid. Van der Waals proposed a correction with two extra terms: one for the volume the molecules themselves occupy, another for the attraction between them. The resulting equation did something the ideal gas law never could: it predicted the existence of a critical point, the temperature above which liquid and vapor stop being distinct phases. The discovery earned van der Waals the Nobel Prize in Physics in 1910.\nThe price of the correction: the elegant linear equation \\(PV=nRT\\) becomes a cubic in \\(V\\),\n$$ \\left(P + \\frac{an^2}{V^2}\\right)(V - nb) = nRT $$But cubics, in practice, have no useful closed-form solution. And this is just one of the equations like this outside physics: from analytical chemistry to pipe engineering, the same kind of problem shows up in different disguises.\nBefore coming back to van der Waals, though, let\u0026rsquo;s start with a warm-up: a quadratic equation, which everyone knows how to solve, and where the interesting question is \u0026ldquo;when is it worth it\u0026rdquo;, not \u0026ldquo;how to solve it\u0026rdquo;. Then comes the cubic from the hook, and finally an equation that is not a polynomial at all.\nWhy Does This Matter in Chemistry and Engineering? # Chemical engineering: equations of state for real gases (such as van der Waals) are the basis of any industrial process design that involves compressing, liquefying or storing gases. Analytical chemistry: calculating the real pH of a solution (without the simplifying classroom shortcuts) matters whenever the dilution is high or the required precision is greater than that of a textbook exercise. Civil and mechanical engineering: sizing any pipe (water, oil, gas) depends on the friction factor, which for decades required manual iteration or reading a chart before scipy.optimize existed. Process industries in general: the three equations in this article show up, in the most varied forms, in any commercial chemical process simulation software. The methods used here, brentq for root finding and numpy.roots for polynomials, are the same ones presented in Solving Equations Numerically with SciPy, the first article of this block. As in the previous article, on physics, here they solve complete chemistry and engineering problems, with real numbers.\n1. Weak Acid pH: When the Classroom Shortcut Is Enough # Every general chemistry course teaches the same recipe for calculating the pH of a weak acid. The dissociation equilibrium,\n$$ HA \\rightleftharpoons H^+ + A^-, \\qquad K_a = \\frac{x^2}{C_0 - x} $$becomes a quadratic equation in \\(x = [H^+]\\). The classroom recipe assumes that the dissociation is small (\\(C_0 - x \\approx C_0\\)) and goes straight to \\(x \\approx \\sqrt{K_a C_0}\\), without solving any quadratic. It is fast and works for most exercises. But it is an approximation, and it is worth seeing how far it strays from the exact answer.\nExample 1: acetic acid, exact vs. approximate Problem statement: A solution of acetic acid (\\(K_a = 1.8 \\times 10^{-5}\\), at 25 °C) has initial concentration \\(C_0 = 0.10\\) mol/L. What is the exact pH, solving the full quadratic, and what is the pH from the general chemistry approximation?\nSolution:\nimport numpy as np from scipy import optimize Ka = 1.8e-5 # acetic acid C0 = 0.10 # mol/L # x^2 + Ka*x - Ka*C0 = 0 roots_acid = np.roots([1, Ka, -Ka * C0]) # only real, positive roots: [H+] cannot be negative or complex x_exact = roots_acid[np.isreal(roots_acid) \u0026amp; (roots_acid.real \u0026gt; 0)].real.max() x_approx = np.sqrt(Ka * C0) pH_exact = -np.log10(x_exact) pH_approx = -np.log10(x_approx) print(f\u0026#34;[H+] exact: {x_exact:.6e} mol/L -\u0026gt; pH = {pH_exact:.4f}\u0026#34;) print(f\u0026#34;[H+] approximate: {x_approx:.6e} mol/L -\u0026gt; pH = {pH_approx:.4f}\u0026#34;) print(f\u0026#34;pH difference: {abs(pH_exact - pH_approx):.5f}\u0026#34;) [H+] exact: 1.332671e-03 mol/L -\u0026gt; pH = 2.8753 [H+] approximate: 1.341641e-03 mol/L -\u0026gt; pH = 2.8724 pH difference: 0.00291For this concentration, the difference is tiny: less than \\(0.003\\) pH unit, irrelevant in practice. The plot below shows why: the curve is the full quadratic, \\(g(x) = x^2 + K_a x - K_a C_0\\), with the exact root marked where it crosses zero. The approximation does not solve this same equation: it is the root of a simpler equation, so it does not land exactly on the curve, but it is visibly close.\nimport matplotlib.pyplot as plt import numpy as np Ka = 1.8e-5 # acetic acid C0 = 0.10 # mol/L roots_acid = np.roots([1, Ka, -Ka * C0]) x_exact = roots_acid[np.isreal(roots_acid) \u0026amp; (roots_acid.real \u0026gt; 0)].real.max() x_approx = np.sqrt(Ka * C0) x_range = np.linspace(0, x_exact * 1.5, 400) def ph_residual(x, Ka, C0): return x**2 + Ka * x - Ka * C0 fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(x_range * 1e3, ph_residual(x_range, Ka, C0), color=\u0026#34;C0\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.scatter([x_exact * 1e3], [0], color=\u0026#34;C2\u0026#34;, zorder=3, label=f\u0026#34;exact root: {x_exact * 1e3:.3f} mmol/L\u0026#34;) ax.axvline(x_approx * 1e3, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, label=f\u0026#34;approximation: {x_approx * 1e3:.3f} mmol/L\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;[H+] (mmol/L)\u0026#34;) ax.set_ylabel(\u0026#34;residual\u0026#34;) ax.set_title(\u0026#34;Weak acid pH: exact root vs. general chemistry approximation\u0026#34;) The exact root and the general chemistry approximation are very close for acetic acid at 0.10 mol/L. When Does the Approximation Stop Holding? # The approximation \\(C_0 - x \\approx C_0\\) is only good when the dissociation is small compared to \\(C_0\\), which fails precisely in very dilute solutions, where a larger fraction of the acid dissociates. By sweeping several concentrations on a logarithmic scale (natural in chemistry, where concentrations vary by orders of magnitude) and comparing exact and approximate pH, it becomes visible where the classroom rule of thumb stops being reliable.\nimport matplotlib.pyplot as plt import numpy as np Ka = 1.8e-5 # acetic acid C0_range = np.logspace(-6, 0, 200) # 1 µmol/L to 1 mol/L pH_exact_range = [] pH_approx_range = [] for C0_i in C0_range: roots_i = np.roots([1, Ka, -Ka * C0_i]) x_exact_i = roots_i[np.isreal(roots_i) \u0026amp; (roots_i.real \u0026gt; 0)].real.max() x_approx_i = np.sqrt(Ka * C0_i) pH_exact_range.append(-np.log10(x_exact_i)) pH_approx_range.append(-np.log10(x_approx_i)) error_pH = np.abs(np.array(pH_exact_range) - np.array(pH_approx_range)) fig, ax = plt.subplots(figsize=(8, 5)) ax.semilogx(C0_range, error_pH, color=\u0026#34;C0\u0026#34;) ax.axhline(0.05, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, label=\u0026#34;0.05 pH unit error\u0026#34;) ax.set_xlabel(r\u0026#34;$C_0$ (mol/L)\u0026#34;) ax.set_ylabel(\u0026#34;|exact pH − approximate pH|\u0026#34;) ax.set_title(\u0026#34;Where the general chemistry approximation breaks down\u0026#34;) ax.legend() The more dilute the solution, the larger the error of the general chemistry approximation: the error crosses 0.05 pH unit near C₀ ≈ 3×10⁻⁴ mol/L. Below approximately \\(3 \\times 10^{-4}\\) mol/L, the error already exceeds \\(0.05\\) pH unit and keeps growing. This is exactly the range of very dilute solutions (common in environmental and trace analyses), where the classroom recipe stops being reliable and the full quadratic becomes worth it.\nThe quadratic is also an approximation This article has been calling the solution of the quadratic \u0026ldquo;exact\u0026rdquo;, as opposed to the classroom shortcut. It is exact within its own model, and that model ignores one thing: the autoprotolysis of water, \\(2H_2O \\rightleftharpoons H_3O^+ + OH^-\\), which also produces \\(H^+\\).\nThe complete model combines the acid equilibrium, the water equilibrium (\\(K_w = 10^{-14}\\)), the mass balance and the charge balance \\([H^+] = [A^-] + [OH^-]\\). The result is a cubic: $$[H^+]^3 + K_a[H^+]^2 - (K_aC_0 + K_w)[H^+] - K_aK_w = 0$$import numpy as np Ka = 1.8e-5 # acetic acid Kw = 1e-14 for C0_i in (1e-1, 1e-4, 1e-6, 1e-7): x_quad = np.roots([1, Ka, -Ka * C0_i]).max() r = np.roots([1, Ka, -(Ka * C0_i + Kw), -Ka * Kw]) x_full = max(r[np.abs(r.imag) \u0026lt; 1e-12].real) print(f\u0026#34;C0={C0_i:.0e}: pH quadratic={-np.log10(x_quad):.4f}\u0026#34; f\u0026#34; pH with water={-np.log10(x_full):.4f}\u0026#34;) C0=1e-01: pH quadratic=2.8753 pH with water=2.8753 C0=1e-04: pH quadratic=4.4638 pH with water=4.4638 C0=1e-06: pH quadratic=6.0223 pH with water=6.0178 C0=1e-07: pH quadratic=7.0024 pH with water=6.7927 For acetic acid at \\(0.10\\) mol/L, nothing changes: water is irrelevant there. But look at the last case, \\(C_0 = 10^{-7}\\) mol/L: the quadratic returns pH \\(7.0024\\), that is, it predicts that an acid solution is slightly basic. This is a physical absurdity, and it comes precisely from ignoring water, which, at this dilution, contributes more \\(H^+\\) than the acid itself. The complete model gives \\(6.79\\), correctly acidic.\nThe lesson holds for the whole article, and is the reason this callout exists: \u0026ldquo;exact\u0026rdquo; is always exact relative to a model. Going down one step of approximation (from the shortcut to the quadratic) solves one problem and reveals the next. The plot above measures the distance between step 1 and step 2; below \\(10^{-6}\\) mol/L, step 3 would be needed.\n2. Van der Waals: Back to the Hook # Back to the equation that opened this article. The molar volume of a real gas has no closed-form solution, but brentq solves it without difficulty, as long as the bracket contains the right physical root.\nExample 2: real CO2 vs. ideal CO2 Problem statement: One mole of CO2 is at \\(P = 10\\) atm and \\(T = 300\\) K. What is the molar volume from the ideal gas law, and what is the real volume from the van der Waals equation (\\(a = 3.640\\) L²·atm/mol², \\(b = 0.04267\\) L/mol)?\nSolution:\nfrom scipy import optimize R = 0.082057366080960 # atm L / (mol K) a_co2, b_co2 = 3.640, 0.04267 # L^2 atm / mol^2 ; L / mol P_val, T_val, n_val = 10.0, 300.0, 1.0 def van_der_waals(V): return (P_val + a_co2 * n_val**2 / V**2) * (V - n_val * b_co2) - n_val * R * T_val V_ideal = n_val * R * T_val / P_val V_vdw = optimize.brentq(van_der_waals, 0.01, 10) print(\u0026#34;V ideal:\u0026#34;, V_ideal, \u0026#34;L\u0026#34;) print(\u0026#34;V van der Waals:\u0026#34;, V_vdw, \u0026#34;L\u0026#34;) print(\u0026#34;deviation:\u0026#34;, (V_vdw - V_ideal) / V_ideal * 100, \u0026#34;%\u0026#34;) V ideal: 2.4617209824287998 L V van der Waals: 2.3524663349745696 L deviation: -4.438140968617679 %Real CO2 occupies about 4.4% less volume than the ideal gas would predict: under these conditions, the attraction between molecules (the term \\(a\\)) slightly outweighs the effect of their own volume (the term \\(b\\)). The plot below shows both solutions: the dashed line is the ideal volume, the dot is the real root found by brentq.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize R = 0.082057366080960 # atm L / (mol K) a_co2, b_co2 = 3.640, 0.04267 # L^2 atm / mol^2 ; L / mol P_val, T_val, n_val = 10.0, 300.0, 1.0 def van_der_waals(V): return (P_val + a_co2 * n_val**2 / V**2) * (V - n_val * b_co2) - n_val * R * T_val V_ideal = n_val * R * T_val / P_val V_vdw = optimize.brentq(van_der_waals, 0.01, 10) V_range = np.linspace(0.05, 3, 500) fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(V_range, van_der_waals(V_range), color=\u0026#34;C0\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.axvline(V_ideal, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, label=\u0026#34;ideal V\u0026#34;) ax.scatter([V_vdw], [0], color=\u0026#34;C2\u0026#34;, zorder=3, label=\u0026#34;van der Waals V\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;V (L)\u0026#34;) ax.set_ylabel(\u0026#34;van der Waals residual\u0026#34;) ax.set_title(\u0026#34;Ideal gas vs. real gas (CO₂)\u0026#34;) The real volume (van der Waals) is about 4.4% below the ideal volume under these conditions. What If the Cubic Has More Than One Real Root? # brentq only finds the root you already expected In Example 2, brentq silently assumed that there was only one physical root in the bracket (0.01, 10). But the van der Waals equation is a real cubic in \\(V\\), and cubics can have up to 3 real roots. brentq never warns that other roots exist outside the given bracket; it simply returns the first one it finds inside it.\nTo find all of them at once, we first need to recognize the cubic hidden in the usual form of the equation. The trick is to multiply everything by \\(V^2\\) and expand:\n$$ \\left(P + \\frac{an^2}{V^2}\\right)(V - nb) = nRT $$$$ PV - Pnb + \\frac{an^2}{V} - \\frac{an^3b}{V^2} = nRT $$Multiplying by \\(V^2\\) and grouping by power of \\(V\\):\n$$ PV^3 - (Pnb + nRT)V^2 + an^2V - an^3b = 0 $$Now it is a degree 3 polynomial with explicit coefficients: exactly the format that numpy.roots expects.\nThree real roots appear below the critical temperature \\(T_c\\), exactly the region of the P-V diagram where liquid and vapor coexist: the \u0026ldquo;loop\u0026rdquo; in the phase diagrams of real gases, and the same discovery that earned van der Waals the Nobel. To find all the roots at once, we rewrite the equation in the explicit polynomial form, \\(PV^3 - (Pnb+nRT)V^2 + an^2V - an^3b = 0\\), and use once again the same numpy.roots from the previous section, the tool that the next article in this series will explore in depth.\nInstead of choosing \\(T\\) and \\(P\\) arbitrarily until the cubic yields 3 roots, we use the law of corresponding states:\nLaw of corresponding states Defining the reduced temperature and pressure \\(T_r = T/T_c\\) and \\(P_r = P/P_c\\) (relative to the critical values \\(T_c\\) and \\(P_c\\) of the substance), any fluid described by the van der Waals equation with \\(T_r \u0026lt; 1\\) has a range of pressures where the cubic admits 3 real roots, regardless of the substance. This is what gives the law its practical usefulness.\nThe caveat matters: this is a property of the equation, not a fact about real gases. Real gases only obey the law of corresponding states approximately, and van der Waals itself is a rather crude equation of state near the critical point; this is why Redlich-Kwong, Soave and Peng-Robinson exist, all more accurate and all still cubic in \\(V\\), that is, solved in exactly the same way.\nimport numpy as np R = 0.082057366080960 # atm L / (mol K) a_co2, b_co2 = 3.640, 0.04267 # L^2 atm / mol^2 ; L / mol T_c = 8 * a_co2 / (27 * R * b_co2) P_c = a_co2 / (27 * b_co2**2) print(f\u0026#34;CO2: T_c = {T_c:.1f} K, P_c = {P_c:.1f} atm\u0026#34;) def van_der_waals_coeffs(P, T, n=1.0): c3 = P c2 = -(P * n * b_co2 + n * R * T) c1 = a_co2 * n**2 c0 = -a_co2 * n**3 * b_co2 return [c3, c2, c1, c0] T_r, P_r = 0.84, 0.41 T_subcritical = T_r * T_c P_subcritical = P_r * P_c print(f\u0026#34;T_r = {T_r} -\u0026gt; T = {T_subcritical:.1f} K\u0026#34;) print(f\u0026#34;P_r = {P_r} -\u0026gt; P = {P_subcritical:.1f} atm\u0026#34;) roots = np.roots(van_der_waals_coeffs(P_subcritical, T_subcritical)) print(\u0026#34;roots (may include complex ones):\u0026#34;, roots) positive_real_roots = np.sort(roots[(np.abs(roots.imag) \u0026lt; 1e-8) \u0026amp; (roots.real \u0026gt; 0)].real) print(\u0026#34;positive real roots, sorted:\u0026#34;, positive_real_roots) CO2: T_c = 308.0 K, P_c = 74.0 atm T_r = 0.84 -\u0026gt; T = 258.7 K P_r = 0.41 -\u0026gt; P = 30.4 atm roots (may include complex ones): [0.53623521 0.13527776 0.07052874] positive real roots, sorted: [0.07052874 0.13527776 0.53623521] Three positive real roots. Physically:\nthe smallest, \\(V \\approx 0.0705\\) L/mol, is the molar volume of the liquid (near the covolume \\(b\\), molecules tightly packed); the largest, \\(V \\approx 0.5362\\) L/mol, is the molar volume of the vapor; the middle one, \\(V \\approx 0.1353\\) L/mol, does not correspond to any stable physical state: it is the part of the curve where \\(\\partial P/\\partial V \u0026gt; 0\\), thermodynamically unstable. \u0026ldquo;Coexist\u0026rdquo; deserves a caveat Saying that the three roots mark the region where liquid and vapor coexist is a convenient simplification, but it is not quite right. Having three real roots means being inside the van der Waals loop, and the loop is wider than the true coexistence.\nEquilibrium coexistence only happens at one specific pressure for each temperature: the saturation pressure, the one at which liquid and vapor have the same chemical potential. It is determined by the Maxwell equal-area construction: draw the horizontal line that cuts the loop so that the two enclosed areas above and below it are equal.\nAway from that specific pressure, the three roots still exist, but what they describe is something else: one of the phases is stable and the other is metastable, that is, superheated liquid or supercooled vapor, states that do exist (it is what lets you superheat water in the microwave), but that collapse at the slightest disturbance. The middle root, on the other hand, describes no state under any condition.\nFor this article, this changes nothing: the three roots are the same, and the point (that mathematics hands over options and physics chooses) still stands. But it is worth knowing that the label \u0026ldquo;coexistence\u0026rdquo; hides one extra geometric construction.\nnumpy.roots knows nothing about thermodynamics: it only returns the 3 mathematical roots of the cubic. Physics, thermodynamic stability, is what decides which of them is the \u0026ldquo;true\u0026rdquo; volume in each phase. It is the same kind of care we saw in the article on polynomials with SymPy (Poly.all_roots() vs. Poly.real_roots()), only now with a concrete physical meaning behind each root. The plot below marks the three roots on the curve, annotating the physical role of each one.\nimport matplotlib.pyplot as plt import numpy as np R = 0.082057366080960 # atm L / (mol K) a_co2, b_co2 = 3.640, 0.04267 # L^2 atm / mol^2 ; L / mol T_c = 8 * a_co2 / (27 * R * b_co2) P_c = a_co2 / (27 * b_co2**2) T_subcritical = 0.84 * T_c P_subcritical = 0.41 * P_c def van_der_waals_coeffs(P, T, n=1.0): c3 = P c2 = -(P * n * b_co2 + n * R * T) c1 = a_co2 * n**2 c0 = -a_co2 * n**3 * b_co2 return [c3, c2, c1, c0] roots = np.roots(van_der_waals_coeffs(P_subcritical, T_subcritical)) positive_real_roots = np.sort(roots[(np.abs(roots.imag) \u0026lt; 1e-8) \u0026amp; (roots.real \u0026gt; 0)].real) V_range_sub = np.linspace(0.05, 1.5, 800) def van_der_waals_sub(V): return (P_subcritical + a_co2 / V**2) * (V - b_co2) - R * T_subcritical V_liquid, V_unstable, V_vapor = positive_real_roots fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(V_range_sub, van_der_waals_sub(V_range_sub), color=\u0026#34;C0\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.scatter([V_liquid, V_unstable, V_vapor], [0, 0, 0], color=\u0026#34;C2\u0026#34;, zorder=3, label=\u0026#34;3 real roots\u0026#34;) ax.annotate(\u0026#34;liquid\u0026#34;, (V_liquid, 0), textcoords=\u0026#34;offset points\u0026#34;, xytext=(-55, 45), ha=\u0026#34;left\u0026#34;, arrowprops=dict(arrowstyle=\u0026#34;-\u0026#34;, color=\u0026#34;gray\u0026#34;, linewidth=1)) ax.annotate(\u0026#34;unstable\u0026#34;, (V_unstable, 0), textcoords=\u0026#34;offset points\u0026#34;, xytext=(40, 55), ha=\u0026#34;center\u0026#34;, arrowprops=dict(arrowstyle=\u0026#34;-\u0026#34;, color=\u0026#34;gray\u0026#34;, linewidth=1)) ax.annotate(\u0026#34;vapor\u0026#34;, (V_vapor, 0), textcoords=\u0026#34;offset points\u0026#34;, xytext=(0, 12), ha=\u0026#34;center\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;V (L/mol)\u0026#34;) ax.set_ylabel(\u0026#34;van der Waals residual\u0026#34;) ax.set_title(f\u0026#34;Van der Waals below T_c: 3 real roots (T={T_subcritical:.0f} K)\u0026#34;) Below T_c, the van der Waals cubic crosses zero three times: only two of these roots correspond to stable physical states. The Isotherms, and What \u0026ldquo;Coexistence\u0026rdquo; Really Requires # All the figures in this section showed the residual \\(f(V)\\), perfect for seeing \u0026ldquo;this is a root problem\u0026rdquo;, but silent about the physics. The canonical figure for the subject is the \\(P\\)-\\(V\\) diagram with isotherms, and it is worth drawing it, because in it the caveat from the previous callout stops being abstract.\nFinding the saturation pressure is itself a root problem, only a nested one. Maxwell\u0026rsquo;s condition is that the horizontal line divides the loop into equal areas:\n$$ \\int_{V_{liq}}^{V_{vap}} \\left[P_{vdW}(V) - P_{sat}\\right] dV = 0 $$And it has an elementary antiderivative, \\(\\int P dV = RT\\ln(V-b) + a/V\\), which avoids numerical quadrature near the singularity at \\(V = b\\):\nimport numpy as np from scipy import optimize R = 0.082057366080960 # atm L / (mol K) a_co2, b_co2 = 3.640, 0.04267 # L^2 atm / mol^2 ; L / mol T_c = 8 * a_co2 / (27 * R * b_co2) P_c = a_co2 / (27 * b_co2**2) T_subcritical = 0.84 * T_c P_subcritical = 0.41 * P_c def van_der_waals_coeffs(P, T, n=1.0): c3 = P c2 = -(P * n * b_co2 + n * R * T) c1 = a_co2 * n**2 c0 = -a_co2 * n**3 * b_co2 return [c3, c2, c1, c0] def P_vdw(V, T): return R * T / (V - b_co2) - a_co2 / V**2 def dPdV_vdw(V, T): return -R * T / (V - b_co2) ** 2 + 2 * a_co2 / V**3 def spinodals(T): \u0026#34;\u0026#34;\u0026#34;The two V where dP/dV = 0: the bottom of the valley and the top of the loop\u0026#39;s hump.\u0026#34;\u0026#34;\u0026#34; return (optimize.brentq(dPdV_vdw, b_co2 * 1.001, 3 * b_co2, args=(T,)), optimize.brentq(dPdV_vdw, 3 * b_co2, 50, args=(T,))) def saturation_pressure(T): V_min, V_max = spinodals(T) # where dP/dV = 0 P_lo, P_hi = P_vdw(V_min, T), P_vdw(V_max, T) # local minimum and maximum of the loop def antiderivative(V): return R * T * np.log(V - b_co2) + a_co2 / V def imbalance(P): r = np.roots(van_der_waals_coeffs(P, T)) r = np.sort(r[np.abs(r.imag) \u0026lt; 1e-9].real) return (antiderivative(r[-1]) - antiderivative(r[0])) - P * (r[-1] - r[0]) return optimize.brentq(imbalance, max(P_lo, 1e-9) * 1.0001, P_hi * 0.9999) P_sat = saturation_pressure(T_subcritical) print(f\u0026#34;P_sat = {P_sat:.2f} atm -\u0026gt; Pr = {P_sat / P_c:.3f}\u0026#34;) P_sat = 35.44 atm -\u0026gt; Pr = 0.479 The example used \\(P_r = 0.41\\), but the equilibrium pressure at that temperature is \\(P_r = 0.479\\). The three roots exist at both pressures, but only at one of them do liquid and vapor truly coexist. At the other, one of the phases is metastable.\nChecking against the literature The van der Waals saturation curve in reduced variables is tabulated, and does not depend on the substance. It serves as a test for the code:\n\\(T_r\\) \\(P_r\\) computed \\(P_r\\) tabulated 0.95 0.8119 0.812 0.90 0.6470 0.647 0.85 0.5045 0.504 0.80 0.3834 0.383 It matches in all four. A word of warning: a brentq that receives a bracket with inverted endpoints does not complain and returns a plausible number (here, \\(P_r = 0.725\\) for \\(T_r = 0.90\\)). Only the comparison with the table gives the error away.\nThe figure below shows, on the left, the classic isotherms (above, at and below \\(T_c\\)) with the critical point marked; and on the right, a zoom into the subcritical loop with the Maxwell construction drawn:\nOn the left, the isotherms: above T_c the curve is monotonic and there is no phase transition; below, the loop appears. On the right, the Maxwell construction: the equilibrium horizontal is the one that makes the two shaded areas equal, and it lies above the pressure used in the example. In the right panel, the two shaded regions have very different shapes (the left one is tall and narrow, the right one is low and wide), but the same area. This equality is what defines \\(P_{sat}\\): the equilibrium horizontal is the only height at which the two compensate each other.\nAlso notice where the bottom of the loop ends up: below zero, down to about \\(-2\\) atm. Van der Waals predicts negative pressure in that range, which describes no real equilibrium state; it is one more sign that the central stretch of the loop is not physics: it is the price of forcing a single smooth equation across a region where, in fact, two phases coexist. The Maxwell construction exists precisely to replace that stretch with the horizontal line.\nNotice the left panel as well: above \\(T_c\\) the isotherm descends with no undulation at all. There is no loop, no three roots, no phase transition. This is exactly what van der Waals predicted and what the ideal gas law could never predict: there is a temperature above which liquid and vapor stop being distinct things.\n3. Colebrook Friction Factor: From the Slide Rule to brentq # Last equation, and the newest of the three, but the one that shows up most in the day-to-day of an engineering project. Published by Cyril Colebrook in 1939, based on experiments he did with Cedric White in 1937 (which is why it is also called the Colebrook-White equation), it gives the friction factor in turbulent flow inside pipes in an implicit form (see the entry Darcy friction factor formulae):\n$$ \\frac{1}{\\sqrt{f}} = -2\\log_{10}\\left(\\frac{\\varepsilon/D}{3.7} + \\frac{2.51}{Re\\sqrt{f}}\\right) $$\\(f\\) appears on both sides, and inside a logarithm: there is no way to isolate it. Five years later, in 1944, Lewis Moody published the chart that would carry his name precisely to spare engineers this calculation: with a ruler and an eye, one read \\(f\\) off a plot instead of iterating by hand. That is how it went, for decades, until programmable calculators. Today, it is a call to brentq, and, later in this section, we rebuild Moody\u0026rsquo;s chart itself from it.\nExample 3: water in a commercial steel pipe Problem statement: Water flows at \\(2\\) m/s (typical design velocity) in a commercial steel pipe with a diameter of \\(100\\) mm. The absolute roughness of commercial steel is \\(\\varepsilon \\approx 0.045\\) mm (a tabulated engineering value); the kinematic viscosity of water at 20 °C is \\(\\nu \\approx 10^{-6}\\) m²/s. What is the friction factor?\nSolution:\nimport numpy as np from scipy import optimize D_pipe = 0.10 # m — 100 mm pipe eps_steel = 0.045e-3 # m — absolute roughness of commercial steel (tabulated) eps_over_D = eps_steel / D_pipe nu_water = 1.0e-6 # m²/s — kinematic viscosity of water at ~20°C v_flow = 2.0 # m/s — typical design velocity in pipes Re = v_flow * D_pipe / nu_water print(f\u0026#34;eps/D = {eps_over_D}\u0026#34;) print(f\u0026#34;Re = {Re:.0f}\u0026#34;) def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) f_friction = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_over_D, Re)) print(\u0026#34;friction factor (brentq):\u0026#34;, f_friction) eps/D = 0.00045 Re = 200000 friction factor (brentq): 0.01856015225418919As in the previous examples, it is worth seeing the curve of the Colebrook equation (rearranged as a residual) crossing zero at the friction factor found.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize D_pipe = 0.10 # m — 100 mm pipe eps_steel = 0.045e-3 # m — absolute roughness of commercial steel (tabulated) eps_over_D = eps_steel / D_pipe nu_water = 1.0e-6 # m²/s — kinematic viscosity of water at ~20°C v_flow = 2.0 # m/s — typical design velocity in pipes Re = v_flow * D_pipe / nu_water def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) f_friction = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_over_D, Re)) f_range = np.linspace(0.008, 0.1, 400) fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(f_range, colebrook(f_range, eps_over_D, Re), color=\u0026#34;C0\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.scatter([f_friction], [0], color=\u0026#34;C2\u0026#34;, zorder=3, label=f\u0026#34;f ≈ {f_friction:.4f}\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;f\u0026#34;) ax.set_ylabel(\u0026#34;Colebrook residual\u0026#34;) ax.set_title(\u0026#34;Friction factor: Colebrook as a root-finding problem\u0026#34;) The Colebrook curve crosses zero at the friction factor found by brentq, f ≈ 0.0186. The Manual Iteration Behind the Phrase \u0026ldquo;Before Calculators\u0026rdquo; # The Colebrook equation is implicit (\\(f\\) appears on both sides), but it can be rearranged by isolating \\(f\\) on the left-hand side:\n$$ f = \\left[-2\\log_{10}\\left(\\frac{\\varepsilon/D}{3.7} + \\frac{2.51}{Re\\sqrt{f}}\\right)\\right]^{-2} $$By guessing an initial value of \\(f\\) and repeatedly substituting from the right-hand side to the left-hand side, the fixed-point iteration, we can solve the equation with no sophisticated method at all. It is exactly what engineers did by hand, or with tables, before calculators and computers.\nimport numpy as np from scipy import optimize D_pipe = 0.10 # m — 100 mm pipe eps_steel = 0.045e-3 # m — absolute roughness of commercial steel (tabulated) eps_over_D = eps_steel / D_pipe nu_water = 1.0e-6 # m²/s — kinematic viscosity of water at ~20°C v_flow = 2.0 # m/s — typical design velocity in pipes Re = v_flow * D_pipe / nu_water def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) f_friction = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_over_D, Re)) def colebrook_fixed_point(f, eps_D, Re_): return (-2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f)))) ** -2 f_current = 0.02 # reasonable initial guess (typical order of magnitude) f_history = [f_current] for _ in range(8): f_current = colebrook_fixed_point(f_current, eps_over_D, Re) f_history.append(f_current) for i, f_i in enumerate(f_history): print(f\u0026#34;iteration {i}: f = {f_i:.8f} error vs. brentq = {abs(f_i - f_friction):.2e}\u0026#34;) iteration 0: f = 0.02000000 error vs. brentq = 1.44e-03 iteration 1: f = 0.01849038 error vs. brentq = 6.98e-05 iteration 2: f = 0.01856372 error vs. brentq = 3.57e-06 iteration 3: f = 0.01855997 error vs. brentq = 1.82e-07 iteration 4: f = 0.01856016 error vs. brentq = 9.28e-09 iteration 5: f = 0.01856015 error vs. brentq = 4.73e-10 iteration 6: f = 0.01856015 error vs. brentq = 2.41e-11 iteration 7: f = 0.01856015 error vs. brentq = 1.23e-12 iteration 8: f = 0.01856015 error vs. brentq = 6.28e-14 The plot below shows this convergence on a logarithmic scale: the error drops roughly linearly with each iteration.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize D_pipe = 0.10 # m — 100 mm pipe eps_steel = 0.045e-3 # m — absolute roughness of commercial steel (tabulated) eps_over_D = eps_steel / D_pipe nu_water = 1.0e-6 # m²/s — kinematic viscosity of water at ~20°C v_flow = 2.0 # m/s — typical design velocity in pipes Re = v_flow * D_pipe / nu_water def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) f_friction = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_over_D, Re)) def colebrook_fixed_point(f, eps_D, Re_): return (-2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f)))) ** -2 f_history = [0.02] for _ in range(8): f_history.append(colebrook_fixed_point(f_history[-1], eps_over_D, Re)) fig, ax = plt.subplots(figsize=(8, 5)) fixed_point_error = [abs(f_i - f_friction) for f_i in f_history] ax.semilogy(range(len(fixed_point_error)), fixed_point_error, \u0026#34;o-\u0026#34;, color=\u0026#34;C0\u0026#34;) ax.set_xlabel(\u0026#34;iteration\u0026#34;) ax.set_ylabel(\u0026#34;|f − f_brentq|\u0026#34;) ax.set_title(\u0026#34;Colebrook fixed-point iteration converging\u0026#34;) The fixed-point iteration converges to the same value as brentq in a few steps, with no need for a bracket or a derivative. It converged to the same value as brentq in a few iterations, with no need for a bracket or a derivative, just repeating the substitution. It is less robust than brentq in general (fixed-point formulas can diverge depending on how the equation is rearranged), but it shows that the \u0026ldquo;magic\u0026rdquo; behind a call to brentq is, deep down, the same manual idea that existed decades before.\nSwamee-Jain: the Explicit Approximation the Industry Uses # There is a third path, between manual iteration and brentq, and it completes the arc. In 1976, Prabhata Swamee and Akalank Jain published an explicit formula that approximates Colebrook with no iteration at all, in a single line of computation:\n$$ f = \\frac{0.25}{\\left[\\log_{10}\\left(\\dfrac{\\varepsilon/D}{3.7} + \\dfrac{5.74}{Re^{0.9}}\\right)\\right]^2} $$import numpy as np from scipy import optimize def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) def swamee_jain(eps_D, Re_): return 0.25 / np.log10(eps_D / 3.7 + 5.74 / Re_**0.9) ** 2 print(f\u0026#34;{\u0026#39;Re\u0026#39;:\u0026gt;8} {\u0026#39;eps/D\u0026#39;:\u0026gt;9} {\u0026#39;brentq\u0026#39;:\u0026gt;9} {\u0026#39;Swamee-Jain\u0026#39;:\u0026gt;12} {\u0026#39;error\u0026#39;:\u0026gt;7}\u0026#34;) for Re_i in (1e4, 2e5, 1e6, 1e7): for eps_i in (1e-7, 0.00045, 0.01): f_exact = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_i, Re_i)) f_approx = swamee_jain(eps_i, Re_i) print(f\u0026#34;{Re_i:\u0026gt;8.0e} {eps_i:\u0026gt;9.5f} {f_exact:\u0026gt;9.5f} {f_approx:\u0026gt;12.5f}\u0026#34; f\u0026#34; {100*(f_approx-f_exact)/f_exact:\u0026gt;+6.2f}%\u0026#34;) Re eps/D brentq Swamee-Jain error 1e+04 0.00000 0.03088 0.03097 +0.29% 1e+04 0.00045 0.03157 0.03175 +0.58% 1e+04 0.01000 0.04313 0.04404 +2.12% 2e+05 0.00000 0.01564 0.01553 -0.68% 2e+05 0.00045 0.01856 0.01867 +0.57% 2e+05 0.01000 0.03821 0.03836 +0.40% 1e+06 0.00000 0.01165 0.01161 -0.33% 1e+06 0.00045 0.01686 0.01695 +0.56% 1e+06 0.01000 0.03796 0.03801 +0.12% 1e+07 0.00000 0.00811 0.00815 +0.50% 1e+07 0.00045 0.01637 0.01640 +0.16% 1e+07 0.01000 0.03791 0.03792 +0.02% Error below \\(1\\%\\) over almost the entire useful range, reaching \\(2\\%\\) only in the most extreme corner (a very rough pipe at low Reynolds number). For sizing a pipe (where the tabulated roughness already has an uncertainty larger than that), it is more than enough, and that is why Swamee-Jain appears in design spreadsheets to this day.\nThis closes a three-step arc that repeats throughout engineering:\nApproach Cost When to use Implicit (Colebrook, 1939) requires iteration it is the reference: defines the \u0026ldquo;right\u0026rdquo; value Manual iterative (fixed point) ~5 substitutions by hand when there is no computer Approximate explicit (Swamee-Jain, 1976) one line, no iteration when \\(1\\%\\) is enough, almost always, in practice brentq did not make the last two obsolete: it made the first one cheap enough to be used directly. But knowing that the others exist explains why so many engineering spreadsheets still call no solver at all.\nMoody Chart # The Colebrook equation only holds for turbulent flow (\\(Re \\gtrsim 4000\\)); in the laminar regime the friction factor has a closed-form formula, \\(f = 64/Re\\). By solving Colebrook via brentq for a range of Reynolds numbers and several relative roughnesses, we rebuild the Moody chart, a classic graphical representation that relates the friction factor, the Reynolds number and the relative roughness, one of the classic figures of fluid engineering.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize D_pipe = 0.10 # m — 100 mm pipe eps_steel = 0.045e-3 # m — absolute roughness of commercial steel (tabulated) eps_over_D = eps_steel / D_pipe nu_water = 1.0e-6 # m²/s — kinematic viscosity of water at ~20°C v_flow = 2.0 # m/s — typical design velocity in pipes Re = v_flow * D_pipe / nu_water def colebrook(f, eps_D, Re_): return 1 / np.sqrt(f) + 2 * np.log10(eps_D / 3.7 + 2.51 / (Re_ * np.sqrt(f))) f_friction = optimize.brentq(colebrook, 1e-4, 0.5, args=(eps_over_D, Re)) Re_range = np.logspace(np.log10(4000), 8, 200) # turbulent regime roughnesses = [0.0, 0.00001, eps_over_D, 0.01] # includes the steel pipe from the example # note on the max(..., 1e-7) that appears below: \u0026#34;smooth pipe\u0026#34; is eps/D = 0, but # then the log10 argument becomes just the Reynolds term — which is legitimate, and # is in fact the smooth limit. The 1e-7 floor exists only so that the eps/D = 0 case # does not produce an exact zero inside the log in any variant of the formula; it is # a value small enough to be indistinguishable from zero in the result fig, ax = plt.subplots(figsize=(9, 6)) for eps_D_i in roughnesses: f_vals = [optimize.brentq(colebrook, 1e-4, 0.5, args=(max(eps_D_i, 1e-7), Re_i)) for Re_i in Re_range] if eps_D_i == 0.0: label = \u0026#34;smooth pipe (ε/D → 0)\u0026#34; elif eps_D_i == eps_over_D: label = f\u0026#34;ε/D = {eps_D_i:g} (example pipe)\u0026#34; else: label = f\u0026#34;ε/D = {eps_D_i:g}\u0026#34; ax.loglog(Re_range, f_vals, label=label) ax.scatter([Re], [f_friction], color=\u0026#34;black\u0026#34;, marker=\u0026#34;*\u0026#34;, s=220, zorder=4, edgecolors=\u0026#34;white\u0026#34;, linewidths=1.2, label=\u0026#34;example (Re, f)\u0026#34;) ax.set_xlabel(\u0026#34;Reynolds number (Re)\u0026#34;) ax.set_ylabel(\u0026#34;friction factor (f)\u0026#34;) ax.set_title(\u0026#34;Moody chart (turbulent regime, via Colebrook + brentq)\u0026#34;) ax.legend(fontsize=10) Moody chart rebuilt via Colebrook + brentq; the example\u0026rsquo;s steel pipe is the highlighted curve, with the computed (Re, f) point marked on it. The example\u0026rsquo;s steel pipe (highlighted curve) is clearly above the smooth pipe: the real roughness of the material noticeably increases friction even in fully established turbulent flow. This is the kind of reading (how the friction factor responds to Reynolds and to roughness, simultaneously) that the Moody chart delivered before any calculator, and that brentq rebuilds here in 800 function calls (4 curves × 200 Reynolds points each).\nSummary Table # Problem Equation Method Result Weak acid pH \\(x^2 + K_a x - K_a C_0 = 0\\) numpy.roots Exact: pH \\(\\approx 2.875\\); approximate: pH \\(\\approx 2.872\\) (difference \\(0.003\\)) Where the approximation fails Same equation, variable \\(C_0\\) repeated numpy.roots Error exceeds \\(0.05\\) pH unit below \\(C_0 \\approx 3\\times10^{-4}\\) mol/L Molar volume of real CO2 \\((P+an^2/V^2)(V-nb)=nRT\\) brentq \\(V \\approx 2.35\\) L, \\(4.4\\%\\) smaller than the ideal volume Liquid-vapor coexistence Same cubic, \\(T_r=0.84\\), \\(P_r=0.41\\) numpy.roots \\(T_c \\approx 308.0\\) K, \\(P_c \\approx 74.0\\) atm; 3 real roots: liquid \\(\\approx 0.071\\), unstable \\(\\approx 0.135\\), vapor \\(\\approx 0.536\\) L/mol Friction factor (Colebrook) \\(1/\\sqrt{f} = -2\\log_{10}(\\ldots)\\) brentq \\(f \\approx 0.0186\\) for a steel pipe, \\(Re=200{,}000\\) Fixed-point iteration Same equation, isolating \\(f\\) Repeated substitution Converges to the brentq value in \\(\\sim\\)5 iterations Moody chart Colebrook for several \\(Re\\) and \\(\\varepsilon/D\\) repeated brentq (800x) Rebuilds the classic chart; example pipe well above the smooth pipe curve Next Steps: End of the Numerical Equations Arc # This article closes the three-part arc on solving equations numerically with SciPy and NumPy. The first article presented the methods (bisection, Brent, Newton, nonlinear systems) in didactic examples; the second applied those methods to physics, from planetary orbits to projectiles with air resistance; this third one took the same methods to chemistry and engineering, always starting from a real numerical scenario, never from an arbitrary value.\nWhat the three articles have in common is the type of problem: always a scalar equation, or a small system, solved point by point. A whole piece is missing: what about when the object of interest is not the root of some equation, but a polynomial, with its own algebra of derivatives, integrals, products and fitting to experimental data? In the next article, Polynomials with NumPy, we see how NumPy treats polynomials as a first-class object, from the same numpy.roots already used in this article to the task that SymPy does not do well: fitting a curve to real noisy data.\n","date":"2026-09-16","externalUrl":null,"permalink":"/2026/09/chemistry-engineering-numerical-equations/","section":"Blog","summary":"From the pH of a weak acid to the volume of a real gas and the friction factor of a pipe: three equations that chemistry and engineering use every day and that only have a useful numerical solution. We solve all three with brentq and numpy.roots and rebuild the Moody chart.","title":"From Chemistry to Engineering: Numerical Equations in Real Systems","type":"posts"},{"content":"","date":"2026-09-16","externalUrl":null,"permalink":"/tags/quimica/","section":"tags","summary":"","title":"química","type":"tags"},{"content":"","date":"2026-09-09","externalUrl":null,"permalink":"/tags/fisica/","section":"tags","summary":"","title":"física","type":"tags"},{"content":" In 1705, the English astronomer Edmond Halley cross-checked records of comets observed in 1531, 1607, and 1682 and reached a bold conclusion: the three could be the same object, returning to the Sun every 76 years or so. Using Kepler\u0026rsquo;s laws (published almost a century earlier) and the mechanics newly formalized by Newton, Halley computed the orbit and predicted the comet\u0026rsquo;s return for 1758. He did not live to check: he died in 1742. But the comet came back at the end of 1758, sixteen years later, and took the name of the man who got the prediction right (see the entry Halley\u0026rsquo;s Comet).\nWho really got the date right Halley\u0026rsquo;s prediction was for 1758, with no month or day: he knew that Jupiter and Saturn would perturb the orbit, but he had no way to calculate by how much. The ones who did that calculation were three French astronomers, between 1757 and November 1758: Alexis Clairaut, Jérôme Lalande, and Nicole-Reine Lepaute, who spent months splitting endless intermediate arithmetic sums among themselves to integrate, step by step, the effect of the two giant planets. They arrived at April 13, 1759, with a margin of one month; the actual perihelion was on March 13.\nIt is one of the great collective numerical calculations of the 18th century, done by hand, in parallel, decades before any machine. Lalande wrote that \u0026ldquo;for six months we calculated from morning to night, sometimes even during meals\u0026rdquo;. This whole article, at bottom, redoes in milliseconds the kind of work that consumed that period.\nThe orbit of Halley\u0026rsquo;s Comet is an extremely elongated ellipse: eccentricity \\(e \\approx 0.967\\) (Earth\u0026rsquo;s is \\(e \\approx 0.017\\), almost a perfect circle). To know where the comet is on any given date, we need to solve Kepler\u0026rsquo;s equation:\n$$ M = E - e\\sin(E) $$where \\(M\\) is the mean anomaly (proportional to the elapsed time) and \\(E\\) is the eccentric anomaly, the angle that actually locates the comet on the ellipse. This equation mixes a free \\(E\\) with \\(\\sin(E)\\): no algebraic manipulation can isolate \\(E\\), and no closed-form formula solves the problem in general. The time has come to really solve it, with numbers.\nWhy Do Orbits and Trajectories Matter? # Aerospace navigation: onboard software on satellites and probes solves Kepler\u0026rsquo;s equation (or variants of it) continuously, to know the exact position at every instant. Ballistics: calculating the range of a projectile under real air resistance (not the frictionless ideal model of high school) is a classic problem of military engineering and, later, of sports such as golf, baseball, and track and field. Orbit determination: from classical celestial mechanics to modern tracking of satellites and space debris, the same kind of equation appears. Physics engines in games and simulations: any simulation of a body in free fall with drag, or in orbit, solves under the hood the same kind of problem as this article. The methods used here, newton and brentq, are exactly the ones presented in Solving Equations Numerically with SciPy, the first article of this block. There, they were applied to didactic examples, chosen to illustrate the behavior of each method. Here, the same methods solve two complete physical problems, with real numbers.\nSolving Kepler\u0026rsquo;s Equation # scipy.optimize.newton solves Kepler\u0026rsquo;s equation quickly when we supply the analytic derivative, \\(1 - e\\cos(E)\\), with quadratic convergence, the same advantage seen in the previous article.\nExample 1: the position of Halley\u0026rsquo;s Comet Statement: At a certain instant, the mean anomaly of Halley\u0026rsquo;s Comet is \\(M = 1.0\\) rad and its orbital eccentricity is \\(e \\approx 0.967\\). What is the eccentric anomaly \\(E\\) at that instant?\nSolution:\nimport numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet M_val = 1.0 E_sol = optimize.newton( kepler_residual, x0=M_val, fprime=kepler_residual_prime, args=(M_val, ecc) ) print(\u0026#34;E (rad):\u0026#34;, E_sol) print(\u0026#34;residual check:\u0026#34;, kepler_residual(E_sol, M_val, ecc)) E (rad): 1.9114369764896801 residual check: 0.0 What Is the Eccentric Anomaly, Anyway? # So far \\(E\\) was just \u0026ldquo;the angle that locates the comet\u0026rdquo;, enough to solve the equation, but not to understand what was solved. And the formulas that come next, \\(x = a(\\cos E - e)\\) and \\(y = a\\sqrt{1-e^2}\\sin E\\), seem to come out of nowhere. It is worth spending a paragraph on the geometric construction, because it makes both formulas obvious.\nDraw, around the ellipse, the auxiliary circle: the circle of radius \\(a\\) (the semi-major axis) centered at the center of the ellipse. Now, given a point \\(P\\) of the planet on the ellipse, go up vertically until you meet the circle, at a point \\(Q\\).\nEccentric anomaly \\(E\\) The eccentric anomaly is the angle of point \\(Q\\) measured from the center of the ellipse (not from the focus), in the direction of the perihelion.\nThe ellipse is literally a circle flattened vertically by a factor \\(b/a = \\sqrt{1-e^2}\\). So, if \\(Q = (a\\cos E,\\ a\\sin E)\\) on the circle, the corresponding point \\(P\\) on the ellipse has the same abscissa and a shrunken ordinate: $$x_{\\text{center}} = a\\cos E, \\qquad y = a\\sqrt{1-e^2}\\sin E$$ Since the Sun is at the focus, not at the center, we shift the origin by \\(ae\\) to the left, and out comes the \\(x = a(\\cos E - e)\\) that we use in the code.\nThe next figure assembles this construction on an orbit of moderate eccentricity (\\(e = 0.6\\), chosen so the drawing stays legible): the dashed auxiliary circle, the vertical projection from \\(Q\\) to \\(P\\), and the two angles marked at their respective vertices (\\(E\\) at the center, \\(\\nu\\) at the focus).\nThe eccentric anomaly E is measured at the center of the ellipse, on the auxiliary circle; the true anomaly ν is measured at the focus, where the Sun is. For E = 65° in this orbit, ν is 103.7°. Three anomalies, and they are easy to confuse The celestial mechanics literature uses three different angles to say \u0026ldquo;where the body is in the orbit\u0026rdquo;, and swapping them is a classic mistake:\nAngle Measured from Used for Mean, \\(M\\) None (not a geometric angle) Grows linearly with time; it is what is computed directly from the date Eccentric, \\(E\\) center of the ellipse Mathematical bridge between \\(M\\) and the position; it is the unknown of Kepler\u0026rsquo;s equation True, \\(\\nu\\) focus (the Sun) It is the angle an observer on the Sun would actually see The mean anomaly is the strangest of the three: it does not correspond to any angle in the drawing. It is a useful fiction: the angle the body would have if it traveled around a circle at constant speed, in the same period. The whole difficulty of the problem is that nature hands us \\(M\\) (proportional to time) and we want \\(\\nu\\) (the real position), and the only path between the two goes through \\(E\\), via the transcendental equation that has no closed form.\nWith that in place, the Cartesian position of the comet from \\(E\\) (with the Sun at the origin, semi-major axis \\(a\\) normalized and eccentricity \\(e\\)) is \\(x = a(\\cos E - e)\\), \\(y = a\\sqrt{1-e^2}\\sin E\\). The next plot shows two panels side by side. On the left, Kepler\u0026rsquo;s equation itself as a root-finding problem, with the curve \\(g(E) = E - e\\sin(E)\\) crossing the horizontal line \\(y=M\\). On the right, what that means physically: the elliptical orbit, the Sun at one of the foci, and the comet at the position corresponding to the \\(E\\) found.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet M_val = 1.0 E_sol = optimize.newton( kepler_residual, x0=M_val, fprime=kepler_residual_prime, args=(M_val, ecc) ) a_orbit = 1.0 # normalized semi-major axis E_range = np.linspace(0, 2 * np.pi, 400) g_E = E_range - ecc * np.sin(E_range) fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(13, 6)) # left panel: Kepler\u0026#39;s equation as a root-finding problem ax1.plot(E_range, g_E, color=\u0026#34;C0\u0026#34;, label=r\u0026#34;$g(E) = E - e\\sin(E)$\u0026#34;) ax1.axhline(M_val, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, label=f\u0026#34;M = {M_val}\u0026#34;) ax1.scatter([E_sol], [M_val], color=\u0026#34;C2\u0026#34;, zorder=3, s=80, label=f\u0026#34;E ≈ {E_sol:.3f} rad\u0026#34;) ax1.set_xlabel(\u0026#34;E (rad)\u0026#34;) ax1.set_ylabel(\u0026#34;g(E)\u0026#34;) ax1.set_title(\u0026#34;Kepler\u0026#39;s equation as a root-finding problem\u0026#34;) ax1.legend() # right panel: the orbit E_orbit = np.linspace(0, 2 * np.pi, 400) x_orbit = a_orbit * (np.cos(E_orbit) - ecc) y_orbit = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_orbit) x_planet = a_orbit * (np.cos(E_sol) - ecc) y_planet = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_sol) ax2.plot(x_orbit, y_orbit, color=\u0026#34;C0\u0026#34;) ax2.scatter([0], [0], color=\u0026#34;C1\u0026#34;, s=200, marker=\u0026#34;*\u0026#34;, zorder=3, label=\u0026#34;Sun (focus)\u0026#34;) ax2.scatter([x_planet], [y_planet], color=\u0026#34;C2\u0026#34;, s=100, zorder=3, label=\u0026#34;comet\u0026#34;) ax2.plot([0, x_planet], [0, y_planet], color=\u0026#34;gray\u0026#34;, linestyle=\u0026#34;:\u0026#34;, linewidth=1) ax2.set_aspect(\u0026#34;equal\u0026#34;) ax2.set_xlabel(\u0026#34;x\u0026#34;) ax2.set_ylabel(\u0026#34;y\u0026#34;) ax2.set_title(f\u0026#34;Elliptical orbit (e = {ecc})\u0026#34;) ax2.legend() Kepler\u0026rsquo;s equation as a root-finding problem (left) and what the root means physically (right): the position of the comet in the orbit. Kepler\u0026rsquo;s Second Law, Finally Visible # So far, we used the elliptical orbit only as a geometric shape, to locate a single point on it. The other half of what Kepler discovered is still missing. The shape of the orbit differs from a circle, and the speed of the comet along it also varies, in a specific way:\nKepler\u0026rsquo;s Second Law (law of areas) The segment joining the Sun to a planet (or comet) sweeps out equal areas in equal intervals of time. Direct consequence: the body moves faster near the perihelion (the point closest to the Sun) and slower near the aphelion (the farthest point).\nSo far, we solved Kepler\u0026rsquo;s equation for a single instant. But the mean anomaly \\(M\\) was defined, by construction, to grow linearly with time; that is exactly why it is easy to compute from any date. This means that sampling \\(M\\) in equal steps is equivalent to sampling time in equal steps. By solving Kepler\u0026rsquo;s equation for several equally spaced \\(M\\) along a complete orbit, we obtain the positions of the comet at equal time intervals, and the second law becomes visible: the points should cluster near the perihelion and be spread out near the aphelion.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet a_orbit = 1.0 # normalized semi-major axis E_orbit = np.linspace(0, 2 * np.pi, 400) x_orbit = a_orbit * (np.cos(E_orbit) - ecc) y_orbit = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_orbit) def orbit_positions(M_samples, ecc): E_samples = np.array([ optimize.newton(kepler_residual, x0=M_i, fprime=kepler_residual_prime, args=(M_i, ecc)) for M_i in M_samples ]) x = a_orbit * (np.cos(E_samples) - ecc) y = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_samples) return x, y n_points = 16 M_samples = np.linspace(0, 2 * np.pi, n_points, endpoint=False) x_samples, y_samples = orbit_positions(M_samples, ecc) fig, ax = plt.subplots(figsize=(7, 7.5)) ax.plot(x_orbit, y_orbit, color=\u0026#34;C0\u0026#34;, zorder=1) ax.scatter([0], [0], color=\u0026#34;C1\u0026#34;, s=200, marker=\u0026#34;*\u0026#34;, zorder=3, label=\u0026#34;Sun (focus)\u0026#34;) ax.scatter(x_samples, y_samples, color=\u0026#34;C2\u0026#34;, s=40, zorder=3, label=\u0026#34;positions (equal Δt)\u0026#34;) for i in range(n_points): ax.plot([0, x_samples[i]], [0, y_samples[i]], color=\u0026#34;gray\u0026#34;, linewidth=0.7, zorder=0) ax.set_aspect(\u0026#34;equal\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.set_title(\u0026#34;Kepler\u0026#39;s second law: denser points near perihelion\u0026#34;) ax.legend(loc=\u0026#34;upper center\u0026#34;, bbox_to_anchor=(0.5, -0.12), ncol=2) Positions at equal time intervals: dense near the perihelion, sparse near the aphelion, the visual signature of Kepler\u0026rsquo;s second law. Visually, we can already notice the clustering near the perihelion (the region of the ellipse close to the Sun, on the right of the plot).\nBut this figure shows the radii, and the law speaks of areas. So it is worth drawing the actual areas: three sectors corresponding to the same time interval (one tenth of the period, each), taken in very different regions of the orbit.\nimport numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet a_orbit = 1.0 # normalized semi-major axis def orbit_positions(M_samples, ecc): E_samples = np.array([ optimize.newton(kepler_residual, x0=M_i, fprime=kepler_residual_prime, args=(M_i, ecc)) for M_i in M_samples ]) x = a_orbit * (np.cos(E_samples) - ecc) y = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_samples) return x, y delta_M = 2 * np.pi / 10 # one tenth of the period sectors = {\u0026#34;near perihelion\u0026#34;: 0.0, \u0026#34;midway\u0026#34;: 2 * np.pi * 0.20, \u0026#34;near aphelion\u0026#34;: 2 * np.pi * 0.45} for label, M0 in sectors.items(): M_sector = np.linspace(M0, M0 + delta_M, 300) xs_s, ys_s = orbit_positions(M_sector, ecc) xs = np.concatenate(([0.0], xs_s)) # closes the polygon at the Sun ys = np.concatenate(([0.0], ys_s)) area = 0.5 * abs(np.dot(xs, np.roll(ys, -1)) - np.dot(ys, np.roll(xs, -1))) print(f\u0026#34;{label:\u0026gt;20}: area = {area:.5f}\u0026#34;) near perihelion: area = 0.08002 midway: area = 0.08004 near aphelion: area = 0.08004 Equal up to the fourth decimal place. And the most impressive part is the contrast between the shapes:\nThe three sectors cover the same time interval and have the same area (0.0800), despite completely different shapes: short and fat near the Sun, long and thin on the opposite side. Near the perihelion, the comet travels a short arc but the distance to the Sun is small, so the sector is fat and low. Near the aphelion, it barely moves, but the radius is huge, so the sector becomes a long, very thin sliver. The two effects compensate each other exactly, and that is what the second law states.\nMeasuring these areas with few samples, however, holds a catch.\nCareful: few points hide the law, they do not refute it Approximating the area of each slice by a triangle with a vertex at the Sun (\\(\\text{area} = \\frac{1}{2}|x_i y_{i+1} - x_{i+1} y_i|\\), with the Sun at the origin) and comparing the 16 slices of the figure above, the variation among them reaches \\(\\mathbf{104\\%}\\). It looks like the second law fails. It does not: near the perihelion, where the comet changes direction fastest, the straight segment joining two consecutive positions \u0026ldquo;cuts the corner\u0026rdquo; relative to the real curved arc, underestimating the area swept there. As we increase the number of samples (each slice now covers a shorter arc of time, and the triangle approximates the curved sector better), the approximation error shrinks and the law is confirmed in practice:\nimport numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet a_orbit = 1.0 # normalized semi-major axis def orbit_positions(M_samples, ecc): E_samples = np.array([ optimize.newton(kepler_residual, x0=M_i, fprime=kepler_residual_prime, args=(M_i, ecc)) for M_i in M_samples ]) x = a_orbit * (np.cos(E_samples) - ecc) y = a_orbit * np.sqrt(1 - ecc**2) * np.sin(E_samples) return x, y def slice_areas(x, y): x_next, y_next = np.roll(x, -1), np.roll(y, -1) return 0.5 * np.abs(x * y_next - x_next * y) for n_test in [16, 100, 1000, 5000]: M_test = np.linspace(0, 2 * np.pi, n_test, endpoint=False) x_test, y_test = orbit_positions(M_test, ecc) areas_test = slice_areas(x_test, y_test) rel_var = (areas_test.max() - areas_test.min()) / areas_test.mean() print(f\u0026#34;n = {n_test:5d} points: relative variation of areas = {rel_var:.2%}\u0026#34;) n = 16 points: relative variation of areas = 104.29% n = 100 points: relative variation of areas = 69.60% n = 1000 points: relative variation of areas = 12.45% n = 5000 points: relative variation of areas = 0.72% With 5000 samples, the slices are already practically equal; the law was right all along, it just needed enough resolution to show up in the numbers.\nNear the Extreme Perihelion: Newton Slows Down # Newton with a derivative converges quickly for moderate eccentricities, but convergence worsens as \\(e \\to 1\\): increasingly elongated orbits, the regime of long-period comets. Solving the same Kepler equation for \\(M=1\\) rad, varying only the eccentricity, and counting how many iterations Newton needs until the residual falls below \\(10^{-12}\\):\nimport matplotlib.pyplot as plt import numpy as np def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) def newton_count_iterations(f, fp, x0, args, tol=1e-12, max_iter=100): x = x0 for i in range(1, max_iter + 1): fx = f(x, *args) if abs(fx) \u0026lt; tol: return i x = x - fx / fp(x, *args) return max_iter # did not converge within this limit ecc = 0.967 # eccentricity of Halley\u0026#39;s Comet M_val = 1.0 eccs = np.linspace(0.1, 0.99, 40) iterations = [ newton_count_iterations(kepler_residual, kepler_residual_prime, M_val, args=(M_val, e_i)) for e_i in eccs ] halley_iter = newton_count_iterations( kepler_residual, kepler_residual_prime, M_val, args=(M_val, ecc) ) print(f\u0026#34;Halley (e={ecc}): {halley_iter} iterations\u0026#34;) fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(eccs, iterations, \u0026#34;o-\u0026#34;, color=\u0026#34;C0\u0026#34;) ax.scatter([ecc], [halley_iter], color=\u0026#34;C1\u0026#34;, zorder=3, s=80, label=f\u0026#34;Halley (e={ecc})\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;eccentricity (e)\u0026#34;) ax.set_ylabel(\u0026#34;iterations to converge\u0026#34;) ax.set_title(\u0026#34;Newton slows down near almost parabolic orbits\u0026#34;) Halley (e=0.967): 7 iterations Newton needs more iterations as the orbit becomes more elongated; Halley\u0026rsquo;s Comet is already on the slowest plateau, with 7 iterations. The relationship is not smooth: the number of iterations rises in steps (from 4 to 5 at \\(e \\approx 0.23\\), from 5 to 6 at \\(e \\approx 0.55\\), from 6 to 7 at \\(e \\approx 0.92\\)), because it is an integer count of steps until crossing the tolerance, not a continuous quantity. Even so, the trend is clear: Halley\u0026rsquo;s Comet, with \\(e \\approx 0.967\\), is already on the slowest plateau observed in this range. Even more elongated orbits (very long-period comets, with \\(e\\) above \\(0.99\\)) would require even more iterations, or an initial guess better than \\(M\\) to speed up convergence.\nWhy Kepler Needed Mars, and Not Earth # The same code answers a historical question that usually goes unnoticed: why did Kepler spend years analyzing data on Mars, and not on the planet right under his feet?\nimport numpy as np from scipy import optimize def kepler_residual(E, M, ecc): return E - ecc * np.sin(E) - M def kepler_residual_prime(E, M, ecc): return 1 - ecc * np.cos(E) def newton_count_iterations(f, fp, x0, args, tol=1e-12, max_iter=100): x = x0 for i in range(1, max_iter + 1): fx = f(x, *args) if abs(fx) \u0026lt; tol: return i x = x - fx / fp(x, *args) return max_iter # did not converge within this limit for name, e_i in [(\u0026#34;Earth\u0026#34;, 0.0167), (\u0026#34;Mars\u0026#34;, 0.0934), (\u0026#34;Halley\u0026#34;, 0.967)]: E_i = optimize.newton(kepler_residual, x0=1.0, fprime=kepler_residual_prime, args=(1.0, e_i)) iterations_i = newton_count_iterations(kepler_residual, kepler_residual_prime, 1.0, args=(1.0, e_i)) print(f\u0026#34;{name:\u0026gt;7} e={e_i:.4f} b/a={np.sqrt(1 - e_i**2):.4f} \u0026#34; f\u0026#34;E={E_i:.4f} rad ({iterations_i} iterations)\u0026#34;) Earth e=0.0167 b/a=0.9999 E=1.0142 rad (3 iterations) Mars e=0.0934 b/a=0.9956 E=1.0825 rad (4 iterations) Halley e=0.9670 b/a=0.2548 E=1.9114 rad (7 iterations) Earth\u0026rsquo;s orbit has \\(b/a = 0.9999\\): it differs from a perfect circle by 0.014%. This flattening is on the order of the error of the best sixteenth century observations (0.5\u0026rsquo; to 1\u0026rsquo; in those of Tycho Brahe): drawn on paper, Earth\u0026rsquo;s orbit is a circle. Mars, with \\(e = 0.0934\\), is still almost circular (\\(b/a = 0.9956\\)), but it has a decisive advantage: since the Sun is at the focus, not at the center, the Sun-Mars distance varies by \\(\\pm 9.3\\%\\) over the Martian year, against \\(\\pm 1.7\\%\\) for Earth.\nIt was this variation, not the flattening, that showed up in Tycho Brahe\u0026rsquo;s data. Kepler spent most of a decade in what he called the \u0026ldquo;war on Mars\u0026rdquo; precisely because Mars was the planet with good data whose orbit departed enough from a circle to give the ellipse away. With Earth, or with Venus (\\(e = 0.007\\)), the discovery simply would not be in the numbers.\nNotice also the iterations column: 3 for Earth, 7 for Halley. It is the same trend that the next section measures in detail.\nProjectile with Air Resistance # The second problem leaves celestial mechanics and returns to Earth: a projectile launched with initial speed \\(v_0\\) and angle \\(\\theta\\), under drag proportional to the speed (\\(F_{\\text{drag}} = -kmv\\)). The height as a function of time has an exponential term:\n$$ y(t) = \\frac{v_{0y} + g/k}{k}\\left(1 - e^{-kt}\\right) - \\frac{gt}{k} $$Finding the flight time, the instant when the height returns to zero, means solving a transcendental equation in \\(t\\).\nWhy linear drag, if it is not the right model here Drag proportional to the speed (\\(F \\propto v\\), called Stokes drag) holds at low Reynolds number: microscopic droplets, spheres falling in a viscous liquid. A projectile at \\(30\\) m/s in air is firmly in the opposite regime, where drag is proportional to the square of the speed (\\(F \\propto v^2\\)).\nSo why use the linear one? Because it is the only one that still has a closed-form trajectory, and that is precisely what enables the comparison in this article: one exact analytic curve against another, with the only unknown (the flight time) falling to a brentq. With quadratic drag there is no closed formula left for the 2D trajectory.\nThe quadratic case, the physically correct one, is solved in Differential Equations with SciPy, and there the tool changes: instead of a transcendental equation for the flight time, we integrate the differential equation in time. Read the numbers below, therefore, as \u0026ldquo;the qualitative effect of drag\u0026rdquo;, not as real ballistics.\nAnd there is a trap before that, exactly the kind the previous article taught us to look for: the height is zero at two instants, at launch and at landing. \\(t = 0\\) is a legitimate root of the equation, and without care it is the one the solver finds. That is why the bracket used next starts at \\(0.1\\) s, not at zero: it deliberately excludes the trivial root.\nA closed form exists, but nobody uses it Technically, it is possible to isolate \\(t\\) in this equation using the Lambert W function, the inverse of \\(f(w) = we^w\\). But this function rarely shows up in introductory courses, and in practice the standard route is numerical anyway, which makes this a legitimate example of \u0026ldquo;no useful closed form\u0026rdquo;, even when a formal solution exists.\nExample 2: flight time with air resistance Statement: A projectile is launched at \\(v_0 = 30\\) m/s, at an angle of \\(45°\\), with drag coefficient \\(k = 0.2\\) s⁻¹. How long does it stay in the air?\nSolution:\nimport numpy as np from scipy import optimize g = 9.81 k_drag = 0.2 # drag coefficient / mass v0 = 30.0 # m/s angle_deg = 45.0 v0x = v0 * np.cos(np.radians(angle_deg)) v0y = v0 * np.sin(np.radians(angle_deg)) def height(t): return (v0y + g / k_drag) / k_drag * (1 - np.exp(-k_drag * t)) - g * t / k_drag flight_time = optimize.brentq(height, 0.1, 20) print(\u0026#34;flight time (with air resistance):\u0026#34;, flight_time, \u0026#34;s\u0026#34;) flight time (with air resistance): 3.838446938469727 sThe next plot shows the height over time, with the root, the flight time, marked where the curve crosses zero again.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize g = 9.81 k_drag = 0.2 # drag coefficient / mass v0 = 30.0 # m/s angle_deg = 45.0 v0x = v0 * np.cos(np.radians(angle_deg)) v0y = v0 * np.sin(np.radians(angle_deg)) def height(t): return (v0y + g / k_drag) / k_drag * (1 - np.exp(-k_drag * t)) - g * t / k_drag flight_time = optimize.brentq(height, 0.1, 20) t_range = np.linspace(0, flight_time * 1.2, 300) fig, ax = plt.subplots(figsize=(8, 5)) ax.plot(t_range, height(t_range), color=\u0026#34;C0\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.scatter([flight_time], [0], color=\u0026#34;C2\u0026#34;, zorder=3, label=\u0026#34;flight time\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;t (s)\u0026#34;) ax.set_ylabel(\u0026#34;height (m)\u0026#34;) ax.set_title(\u0026#34;Projectile with linear air resistance\u0026#34;) Height of the projectile over time: the root of the curve is the flight time. How Much Does Air Resistance Really Change the Trajectory? # The ideal case (no drag) has a closed-form trajectory and flight time: \\(t_{\\text{flight}}^{\\text{ideal}} = 2v_{0y}/g\\). Overlaying the two complete trajectories (position \\(x(t)\\) against \\(y(t)\\), not just height against time), it becomes visible how much drag reduces the range.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize g = 9.81 k_drag = 0.2 # drag coefficient / mass v0 = 30.0 # m/s angle_deg = 45.0 v0x = v0 * np.cos(np.radians(angle_deg)) v0y = v0 * np.sin(np.radians(angle_deg)) def height(t): return (v0y + g / k_drag) / k_drag * (1 - np.exp(-k_drag * t)) - g * t / k_drag flight_time = optimize.brentq(height, 0.1, 20) ideal_flight_time = 2 * v0y / g t_range_ideal = np.linspace(0, ideal_flight_time, 300) x_ideal = v0x * t_range_ideal y_ideal = v0y * t_range_ideal - 0.5 * g * t_range_ideal**2 t_range_drag = np.linspace(0, flight_time, 300) x_drag = v0x / k_drag * (1 - np.exp(-k_drag * t_range_drag)) y_drag = height(t_range_drag) fig, ax = plt.subplots(figsize=(9, 5)) ax.plot(x_ideal, y_ideal, color=\u0026#34;C0\u0026#34;, label=\u0026#34;without air resistance (closed form)\u0026#34;) ax.plot(x_drag, y_drag, color=\u0026#34;C1\u0026#34;, label=\u0026#34;with air resistance (numerical)\u0026#34;) ax.scatter([x_ideal[-1]], [0], color=\u0026#34;C0\u0026#34;, zorder=3) ax.scatter([x_drag[-1]], [0], color=\u0026#34;C1\u0026#34;, zorder=3) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.set_ylim(0, y_ideal.max() * 1.4) # room at the top so the legend does not cover the curves ax.legend(loc=\u0026#34;upper right\u0026#34;) ax.set_xlabel(\u0026#34;x (m)\u0026#34;) ax.set_ylabel(\u0026#34;y (m)\u0026#34;) ax.set_title(\u0026#34;Trajectory: with × without air resistance\u0026#34;) Trajectory with and without air resistance: drag reduces both the range and the flight time. import numpy as np from scipy import optimize g = 9.81 k_drag = 0.2 # drag coefficient / mass v0 = 30.0 # m/s angle_deg = 45.0 v0x = v0 * np.cos(np.radians(angle_deg)) v0y = v0 * np.sin(np.radians(angle_deg)) def height(t): return (v0y + g / k_drag) / k_drag * (1 - np.exp(-k_drag * t)) - g * t / k_drag flight_time = optimize.brentq(height, 0.1, 20) ideal_flight_time = 2 * v0y / g ideal_range = v0x * ideal_flight_time drag_range = v0x / k_drag * (1 - np.exp(-k_drag * flight_time)) print(f\u0026#34;ideal range: {ideal_range:.1f} m (flight_time = {ideal_flight_time:.2f} s)\u0026#34;) print(f\u0026#34;range with air resistance: {drag_range:.1f} m (flight_time = {flight_time:.2f} s)\u0026#34;) ideal range: 91.7 m (flight_time = 4.32 s) range with air resistance: 56.8 m (flight_time = 3.84 s) With \\(k = 0.2\\), drag cuts the range by about 38% (from \\(91.7\\) m to \\(56.8\\) m) and reduces the flight time by half a second. The lesson lies more in the order of magnitude of the error than in the exact number (which depends on the linear model, as warned above): ignoring air resistance produces a deviation of tens of percent, not a few. That is why the high school parabola, however elegant it may be, is of no use for any practical ballistics application, and why it is worth learning to solve the resisted case, even when it costs a numerical root instead of a formula.\nHow Much Does the Drag Coefficient Matter? # k_drag = 0.2 was a single value, chosen for the example above. This coefficient depends on the shape of the projectile, frontal area, air density, and, crucially, mass: the \\(k\\) that appears in the formulas is the ratio between the raw drag coefficient and the mass of the projectile, \\(k = b/m\\), with unit s⁻¹. That is why a tennis ball and a cannonball of the same size behave so differently: same frontal area, same \\(b\\), very different masses, and therefore very different \\(k\\). By repeating the same brentq over a range of \\(k\\) values, we can see how the range and the flight time behave without needing any new tool.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize g = 9.81 k_drag = 0.2 # drag coefficient / mass v0 = 30.0 # m/s angle_deg = 45.0 v0x = v0 * np.cos(np.radians(angle_deg)) v0y = v0 * np.sin(np.radians(angle_deg)) def height(t): return (v0y + g / k_drag) / k_drag * (1 - np.exp(-k_drag * t)) - g * t / k_drag flight_time = optimize.brentq(height, 0.1, 20) drag_range = v0x / k_drag * (1 - np.exp(-k_drag * flight_time)) def height_with_k(t, k): return (v0y + g / k) / k * (1 - np.exp(-k * t)) - g * t / k def range_with_k(k): flight_time_k = optimize.brentq(height_with_k, 0.1, 20, args=(k,)) range_k = v0x / k * (1 - np.exp(-k * flight_time_k)) return range_k, flight_time_k k_range = np.linspace(0.02, 0.6, 40) results = [range_with_k(k_i) for k_i in k_range] ranges = [r[0] for r in results] flight_times = [r[1] for r in results] fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5)) ax1.plot(k_range, ranges, color=\u0026#34;C0\u0026#34;) ax1.scatter([k_drag], [drag_range], color=\u0026#34;C2\u0026#34;, zorder=3, label=f\u0026#34;k = {k_drag}\u0026#34;) ax1.set_xlabel(\u0026#34;drag coefficient (k)\u0026#34;) ax1.set_ylabel(\u0026#34;range (m)\u0026#34;) ax1.set_title(\u0026#34;Range vs. drag\u0026#34;) ax1.legend() ax2.plot(k_range, flight_times, color=\u0026#34;C1\u0026#34;) ax2.scatter([k_drag], [flight_time], color=\u0026#34;C2\u0026#34;, zorder=3, label=f\u0026#34;k = {k_drag}\u0026#34;) ax2.set_xlabel(\u0026#34;drag coefficient (k)\u0026#34;) ax2.set_ylabel(\u0026#34;flight time (s)\u0026#34;) ax2.set_title(\u0026#34;Flight time vs. drag\u0026#34;) ax2.legend() Range and flight time fall smoothly as the drag coefficient increases: from 86.7 m (k almost zero) to 30.5 m (k = 0.6). At the extremes tested, the range falls from \\(86.7\\) m (\\(k = 0.02\\), almost negligible drag) to \\(30.5\\) m (\\(k = 0.6\\), strong drag), less than a third. The drop is smooth and monotonic: there is no qualitative surprise here, unlike what happened with Kepler\u0026rsquo;s eccentricity (where steps appeared). It makes sense: more drag always means less range, with no special threshold behavior.\nSummary Table # Problem Equation Method Result Position of Halley\u0026rsquo;s Comet \\(M = E - e\\sin(E)\\) newton (with derivative) \\(E \\approx 1.911\\) rad for \\(M=1\\), \\(e \\approx 0.967\\) Kepler\u0026rsquo;s second law Areas of the slices between positions newton (several samples) + numerical check Relative variation drops from 104% (n=16) to 0.72% (n=5000) Convergence near \\(e \\to 1\\) Same Kepler equation Iteration count of newton From 4 to 7 iterations in the range \\(e \\in [0.1, 0.99]\\) Flight time with linear drag \\(y(t) = 0\\), exponential form brentq \\(t_{\\text{flight}} \\approx 3.84\\) s, range \\(\\approx 56.8\\) m (against \\(91.7\\) m in the ideal case without drag) Sensitivity to drag Same equation, variable \\(k\\) repeated brentq Range from \\(86.7\\) m to \\(30.5\\) m for \\(k \\in [0.02, 0.6]\\) Next Steps # The two problems in this article came from physics (one from celestial mechanics, the other from ballistics) and each used a single real numerical scenario (Halley\u0026rsquo;s Comet, a launch at \\(45°\\)) instead of isolated didactic examples. But physics is not the only domain where equations with no closed form show up all the time. Chemistry and engineering have their own, with different traps: cubics with multiple physical roots, classroom approximations that stop holding, and implicit equations that engineers solved by hand decades before scipy.optimize existed.\nIn the next article, From Chemistry to Engineering, we solve three of these real equations (the molar volume of a real gas, the pH of a weak acid, and the Colebrook friction factor), all with the same tools presented in this block.\n","date":"2026-09-09","externalUrl":null,"permalink":"/2026/09/orbits-trajectories-physics/","section":"Blog","summary":"Where is Halley’s Comet on any given date? The answer requires solving Kepler’s equation, which has no closed form. With SciPy’s newton and brentq, we locate the comet, make the law of areas visible, and compute the range of a projectile with air resistance.","title":"Orbits and Trajectories: Numerical Equations in Physics","type":"posts"},{"content":"","date":"2026-09-09","externalUrl":null,"permalink":"/tags/physics/","section":"tags","summary":"","title":"physics","type":"tags"},{"content":" In 1609, the German astronomer Johannes Kepler published two laws of planetary motion. The first went against tradition: orbits are not perfect circles, as Ptolemy and even Copernicus believed, but ellipses, with the Sun at one of the foci. The second, the \u0026ldquo;law of areas,\u0026rdquo; said that a planet sweeps out equal areas in equal times along its orbit: faster near the Sun, slower far from it.\nBeautiful from a geometric point of view. But it hid an annoying practical problem. If an astronomer wanted to know exactly where a planet would be at a specific instant (not \u0026ldquo;at some point,\u0026rdquo; but at 2:32 p.m. on a given date), they had to solve an equation. That equation, later named Kepler\u0026rsquo;s equation, is:\n$$ M = E - e \\sin(E) $$where \\(M\\) is the mean anomaly (proportional to the elapsed time, easy to compute), \\(e\\) is the orbit\u0026rsquo;s eccentricity (how \u0026ldquo;flattened\u0026rdquo; the ellipse is) and \\(E\\) is the eccentric anomaly, the angle that actually connects time to the planet\u0026rsquo;s position on the orbit. Given \\(M\\) and \\(e\\), we want \\(E\\).\nThe problem: this equation mixes a free \\(E\\) with \\(\\sin(E)\\). There is no algebraic manipulation capable of isolating \\(E\\) on one side. Many astronomers, from Kepler onward, developed methods and tables to deal with this equation, long before \u0026ldquo;numerical method\u0026rdquo; was a formal term.\nDoes any modern tool solve this exactly? Let\u0026rsquo;s let SymPy try:\nimport sympy as sp E, M, e = sp.symbols(\u0026#34;E M e\u0026#34;) kepler_eq = sp.Eq(M, E - e * sp.sin(E)) sp.solve(kepler_eq, E) NotImplementedError: multiple generators [E, sin(E)] No algorithms are implemented to solve equation M + (-E + e*sin(E)) Not even SymPy, with all its symbolic power, can handle it, and the reason is not that the library is \u0026ldquo;weak\u0026rdquo;: it is the nature of the equation that gets in the way. It mixes a polynomial term (\\(E\\)) with a transcendental function (\\(\\sin E\\)) in the same unknown, and there is no closed-form formula, in terms of elementary functions, that solves this in general.\nThis is the first of three articles about one of those solutions: solving numerically, with SciPy, equations that SymPy cannot solve symbolically.\nWhy Does Solving Equations Numerically Matter? # Equations with no closed-form solution show up all the time in real problems, and Kepler\u0026rsquo;s is just one of them:\nEngineering: the friction factor for fluid flow in pipes (the Colebrook equation) is defined implicitly, and decades of engineers solved it by hand iteration before scipy.optimize existed. Chemistry: equations of state for real gases (such as van der Waals) and full chemical equilibria, without the simplifying textbook approximations, become polynomials or transcendental equations with no practical closed form. Astronomy and aerospace: Kepler\u0026rsquo;s equation itself is solved numerically in orbit propagation software for satellites and space probes. Finance: the internal rate of return (IRR) of a cash flow is a root of a polynomial and, for high degrees, there is no guarantee of a closed form (see the note below). Data science and optimization: finding critical points of complex loss functions is, at bottom, solving \u0026ldquo;derivative equals zero,\u0026rdquo; an equation that rarely has a closed-form solution in nontrivial models. No general formula from degree 5 onward In 1824, the Norwegian mathematician Niels Henrik Abel proved that there is no general formula, involving only radicals, that solves every polynomial equation of degree 5 or higher: the Abel-Ruffini theorem. In other words, even restricting ourselves to purely polynomial problems (no sines or exponentials), the absence of a closed-form solution is the rule from a certain degree on. Numerical methods are often the only tool that exists.\nThe Italian Paolo Ruffini published a first proof in 1799, with a gap; Abel presented a complete proof in 1824. But it was the Frenchman Évariste Galois who, around 1832, answered the bigger question: not only that there is no general formula, but which specific equations are solvable by radicals and which are not. The answer depends on the symmetry structure of the roots, giving rise to group theory. Galois died in a duel at age 20, and his manuscripts were only understood more than a decade later.\nWe have already written plenty on this site about SymPy, the symbolic library for Python: it solves equations and systems exactly whenever a closed-form solution exists. SciPy comes in exactly where SymPy stops: when there is no formula, it approximates the answer numerically, with error control and (almost always) lots of speed.\nFamiliar Ground, Now in Numbers # Before attacking what SymPy cannot solve, it is worth seeing how SciPy does on problems that SymPy already solves exactly, so we can compare directly. And since NumPy will appear throughout the article alongside SciPy, it is worth explaining why before moving on.\nNumPy and SciPy are not competitors SciPy is built on top of NumPy: it uses the same array (ndarray) as its central data structure and depends on it to work. The division of labor, historically, ended up like this: NumPy takes care of the array itself (elementary operations, a bit of linear algebra and Fourier transforms, random number generation, and a few generic comparison utilities). SciPy adds domain-specialized algorithms, organized in subpackages; scipy.optimize, the subject of this article, is one of them. SciPy never reimplements what NumPy already does well, so it is normal for the two to appear side by side in the same piece of code.\nThere is also a real point of overlap: numpy.linalg.solve (used in Example 2 just below) and scipy.linalg.solve do essentially the same thing, and the SciPy version is usually the more recommended one when both are installed (more options, always via LAPACK). We use NumPy\u0026rsquo;s here only because it is the better known; for a simple system like this one, it makes no practical difference.\nThe table below summarizes where each tool used in this article comes from, and why:\nTool Comes from Role numpy.roots NumPy Roots of a polynomial (via eigenvalues of the companion matrix) numpy.linalg.solve NumPy Linear system (scipy.linalg.solve is equivalent) numpy.isclose / numpy.allclose NumPy Float comparison with tolerance numpy.sign, numpy.diff, numpy.linspace NumPy Basic array operations scipy.optimize.bisect / brentq SciPy Root finding by bracket scipy.optimize.newton SciPy Newton\u0026rsquo;s method scipy.optimize.fsolve / root SciPy Systems of nonlinear equations Example 1: the same quadratic from the equations article Statement: In the article Solving Equations with SymPy, we solved \\(x^2 + 2x - 8 = 0\\) with sympy.solve, obtaining the exact roots \\(-4\\) and \\(2\\). What does the same problem look like solved numerically?\nSolution: The numerical equivalent is numpy.roots, which takes the coefficients from the highest degree to the lowest, the same convention as SymPy\u0026rsquo;s all_coeffs():\nimport numpy as np print(np.roots([1, 2, -8])) [-4. 2.] Same result, now as float64 instead of SymPy\u0026rsquo;s exact integers. For a simple quadratic like this, the difference is cosmetic. Later on it stops being so.\nExample 2: the same linear system Statement: The same article solved the system \\(x + y = 3\\), \\(3x - 2y = 0\\) with sympy.solve, obtaining \\(x = 6/5\\), \\(y = 9/5\\), exactly. And numerically?\nSolution: In matrix form \\(A\\mathbf{x} = \\mathbf{b}\\), the numerical equivalent is numpy.linalg.solve:\nimport numpy as np A = np.array([[1.0, 1.0], [3.0, -2.0]]) b = np.array([3.0, 0.0]) linear_sol = np.linalg.solve(A, b) print(linear_sol) [1.2 1.8] Once again, the same result (\\(1.2 = 6/5\\) and \\(1.8 = 9/5\\)), only as a floating-point approximation.\nCareful with floats (revisiting the previous articles) In the original article, comparing the solution with == after substituting 6/5 as a float gave False on one of the equations and only worked with Rational (see why). Here the situation is reversed: we are deliberately working in floating point, so the \u0026ldquo;is this solution correct?\u0026rdquo; check must never use == against zero.\nimport numpy as np A = np.array([[1.0, 1.0], [3.0, -2.0]]) b = np.array([3.0, 0.0]) linear_sol = np.linalg.solve(A, b) residual = A @ linear_sol - b print(\u0026#34;residual A@x - b:\u0026#34;, residual) print(\u0026#34;is it zero, within tolerance?\u0026#34;, np.allclose(residual, 0)) print(\u0026#34;is it exactly zero? (don\u0026#39;t do this)\u0026#34;, np.all(residual == 0)) residual A@x - b: [4.4408921e-16 0.0000000e+00] is it zero, within tolerance? True is it exactly zero? (don\u0026#39;t do this) False The residual is practically zero (\\(4.4 \\times 10^{-16}\\), the order of magnitude of the rounding error of double-precision floating point), but comparing exactly against zero fails because one of the entries carries a bit of rounding residue. From here on, every \u0026ldquo;is this a solution?\u0026rdquo; check in this article (and in the next two) uses numpy.isclose (compares two numbers and tells whether the difference between them is smaller than a tolerance) or its version for whole arrays, numpy.allclose.\nThe Idea of Bisection: bisect and brentq # Back to the real problem: equations with no closed form, like \\(e^x = x^{100}\\). Yes, this equation has already appeared around here, in the article on limits, when we compared the growth of an exponential against that of a power. There, SymPy found one of the roots, \\(647.2775124394\\), already in numerical form. The exact form of this root exists, but it uses the Lambert W function, which is not elementary.\nThe simplest and most robust idea for finding roots numerically comes straight from the Intermediate Value Theorem, probably already seen in any calculus course:\nIntermediate Value Theorem (and the idea of the bracket) If \\(f\\) is continuous on \\([a, b]\\) and \\(f(a)\\) and \\(f(b)\\) have opposite signs, then there exists at least one \\(c \\in (a, b)\\) such that \\(f(c) = 0\\).\nAn interval \\([a, b]\\) where \\(f\\) changes sign is called a bracket, the term the numerical methods literature uses to designate exactly this interval that \u0026ldquo;contains\u0026rdquo; the root. It is the minimum guarantee that a root exists in there, and it is exactly this guarantee that bisection exploits: split the interval in half, see on which side the sign changes, discard the other half, repeat.\nSciPy implements this idea to the letter in scipy.optimize.bisect, with no extra cleverness, just the interval splitting repeated until the desired tolerance. Let\u0026rsquo;s apply it to the same \\(f(x) = x^2 - 2\\) that reappears later in this article, looking for \\(\\sqrt{2}\\) in the bracket \\([1, 2]\\) (\\(f(1) = -1\\), \\(f(2) = 2\\), opposite signs, hence a valid bracket):\nfrom scipy import optimize def f(x): return x**2 - 2 print(optimize.bisect(f, 1.0, 2.0)) 1.4142135623715149 Correct to the 11th decimal place, compared with the exact value \\(\\sqrt{2} \\approx 1.4142135623730951\\). The error is \\(1.6 \\times 10^{-12}\\), exactly the order of bisect\u0026rsquo;s default tolerance. It works and is guaranteed, but it is slow: each iteration only halves the interval (the section on orders of convergence, later in this article, shows exactly how much slower).\nIt is worth opening the black box and watching the first four iterations by hand. The rule is always the same: compute the midpoint \\(m\\), check whether \\(f(m)\\) has the same sign as \\(f(a)\\), and discard the half where the root cannot be:\nIteration Interval \\([a, b]\\) \\(m = \\frac{a+b}{2}\\) \\(f(m)\\) Discarded half New width 1 [1.0000, 2.0000] \\(1.5000\\) \\(+0.2500\\) [1.5000, 2.0000] \\(0.5000\\) 2 [1.0000, 1.5000] \\(1.2500\\) \\(-0.4375\\) [1.0000, 1.2500] \\(0.2500\\) 3 [1.2500, 1.5000] \\(1.3750\\) \\(-0.1094\\) [1.2500, 1.3750] \\(0.1250\\) 4 [1.3750, 1.5000] \\(1.4375\\) \\(+0.0664\\) [1.4375, 1.5000] \\(0.0625\\) Since \\(f(1) = -1\\) is negative, whenever \\(f(m)\\) is also negative the root is on the right side of \\(m\\), and the left half goes out, and vice versa. After 4 iterations we know that \\(\\sqrt{2}\\) lies between \\(1.3750\\) and \\(1.4375\\), and indeed it does. Note the last column: the width drops by an exact factor of 2 at each step, always, regardless of the function. That is the guarantee bisection offers, and only that.\nFor most cases, SciPy offers a turbocharged version of the same idea: the function scipy.optimize.brentq, which uses Brent\u0026rsquo;s method, a combination of bisection with inverse quadratic interpolation that preserves the convergence guarantee of bisection but is usually much faster in practice.\nLet\u0026rsquo;s apply it to the same equation from the limits article. But there is one detail: the function \\(f(x) = e^x - x^{100}\\) ranges from \\(10^0\\) to \\(10^{304}\\) over this range of \\(x\\), and an ordinary plot of the value of \\(f(x)\\) is visually useless (a line flattened at zero until almost the end). To choose a bracket, what matters is the sign of \\(f(x)\\), not its magnitude, so the sign is what we plot. The following code generates the figure right below:\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize def f(x): return np.exp(x) - x**100 root = optimize.brentq(f, 600, 700) x_vals = np.linspace(600, 700, 2000) sign_vals = np.sign(f(x_vals)) fig, ax = plt.subplots(figsize=(8, 5)) ax.step(x_vals, sign_vals, color=\u0026#34;C0\u0026#34;, where=\u0026#34;post\u0026#34;) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) ax.axvline(root, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, label=f\u0026#34;root ≈ {root:.2f}\u0026#34;) ax.set_yticks([-1, 0, 1]) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(r\u0026#34;sign of $e^x - x^{100}$\u0026#34;) ax.set_title(\u0026#34;Choosing the bracket for brentq: where the sign changes\u0026#34;) ax.legend() Choosing the bracket for brentq: what matters is where the sign changes, not the raw value of the function. But how do we arrive at a plot like that without already knowing where to look? In practice, nobody \u0026ldquo;guesses\u0026rdquo; a bracket: it comes from knowledge of the problem (reasonable physical limits, for example) or from a simple computational scan, in which the function is evaluated on a set of evenly spaced points and we look for where the sign changes from one point to the next.\nimport numpy as np def f(x): return np.exp(x) - x**100 x_scan = np.linspace(600, 700, 1000) signs = np.sign(f(x_scan)) changes = np.where(np.diff(signs) != 0)[0] print(\u0026#34;sign changes between x =\u0026#34;, x_scan[changes], \u0026#34;and x =\u0026#34;, x_scan[changes + 1]) sign changes between x = [647.24724725] and x = [647.34734735] numpy.diff computes the difference between consecutive elements of the array of signs. Where that difference is nonzero, the sign changed from one point to the next, and there is the bracket we were after: (647.247..., 647.347...), right around the already known root \\(647.28\\). This same logic (scan, compare consecutive signs, refine the interval) is what more sophisticated root-finding algorithms automate under the hood.\nimport numpy as np from scipy import optimize def f(x): return np.exp(x) - x**100 root = optimize.brentq(f, 600, 700) print(\u0026#34;root found:\u0026#34;, root) root found: 647.2775124394004 The same value as in the limits article, now obtained with explicit control over the method used. But brentq is not magic: it requires the bracket. An interval without a sign change fails explicitly, instead of silently returning something wrong:\nimport numpy as np from scipy import optimize def f(x): return np.exp(x) - x**100 optimize.brentq(f, 2, 5) ValueError: f(a) and f(b) must have different signs Careful: not every interval \u0026ldquo;looks\u0026rdquo; safe (0, 10) would not be a good counterexample here: there is a much smaller root in that range, \\(x \\approx 1.01\\) (also mentioned in the limits article), so the sign changes there and brentq would work normally. The error above only shows up because I chose an interval, (2, 5), where \\(f\\) stays negative the whole time, between the two neighboring roots. Always plot the function (or, as here, its sign) before choosing a bracket.\nNewton\u0026rsquo;s Method # Bisection (and brentq, by extension) is robust but \u0026ldquo;blind\u0026rdquo;: it only uses the sign of the function, ignoring all the information about how fast it is changing. Newton\u0026rsquo;s method (or Newton-Raphson) uses the derivative to take much smarter steps and, in exchange, loses the unconditional guarantee of convergence.\nNewton\u0026rsquo;s method Given an estimate \\(x_n\\) for the root of \\(f\\), draw the tangent line to the graph of \\(f\\) at \\(x_n\\) and use the point where that tangent crosses the \\(x\\) axis as the next estimate: $$x_{n+1} = x_n - \\frac{f(x_n)}{f'(x_n)}$$ Repeat until the value converges within a desired tolerance.\nThis formula does not need to be memorized: it comes out in two lines. The tangent line to the graph of \\(f\\) at the point \\(x_n\\) is\n$$ y = f(x_n) + f'(x_n)(x - x_n) $$We want where this line crosses the \\(x\\) axis, that is, where \\(y = 0\\):\n$$ 0 = f(x_n) + f'(x_n)(x - x_n) \\quad\\Longrightarrow\\quad x = x_n - \\frac{f(x_n)}{f'(x_n)} $$And this \\(x\\) is precisely the next guess, \\(x_{n+1}\\). The whole formula is this: replace the function with its tangent and solve the easy problem in place of the hard one. It becomes clear, in passing, where the method\u0026rsquo;s fragility comes from: if \\(f^{\\prime}(x_n)\\) is zero, the tangent is horizontal and never crosses the axis; if it is nearly zero, it crosses very far away. That is exactly what we are about to see fail.\nBefore treating scipy.optimize.newton as a black box, it is worth rebuilding by hand what it does, in the most classic example possible: finding \\(\\sqrt{2}\\) by solving \\(f(x) = x^2 - 2 = 0\\).\nimport numpy as np def f(x): return x**2 - 2 def fp(x): return 2 * x def newton_manual(f, fp, x0, n_iter): xs = [x0] for _ in range(n_iter): x_curr = xs[-1] xs.append(x_curr - f(x_curr) / fp(x_curr)) return xs xs = newton_manual(f, fp, x0=1.0, n_iter=4) for i, x_i in enumerate(xs): print(f\u0026#34;x_{i} = {x_i:.10f} (error: {abs(x_i - np.sqrt(2)):.2e})\u0026#34;) x_0 = 1.0000000000 (error: 4.14e-01) x_1 = 1.5000000000 (error: 8.58e-02) x_2 = 1.4166666667 (error: 2.45e-03) x_3 = 1.4142156863 (error: 2.12e-06) x_4 = 1.4142135624 (error: 1.59e-12) Note the speed: the number of correct digits practically doubles at each step (from 1 to 2, to 5, to 12). This is the so-called quadratic convergence of Newton\u0026rsquo;s method, and it is the reason it is so popular when the derivative is available. The figure below shows the first 3 steps visually, with the tangent lines converging rapidly to \\(\\sqrt{2}\\) (from the fourth step on, the tangent is already too close to the root to be distinguishable in the drawing).\nimport matplotlib.pyplot as plt import numpy as np def f(x): return x**2 - 2 def fp(x): return 2 * x def newton_manual(f, fp, x0, n_iter): xs = [x0] for _ in range(n_iter): x_curr = xs[-1] xs.append(x_curr - f(x_curr) / fp(x_curr)) return xs xs_plot = newton_manual(f, fp, x0=1.0, n_iter=2) # x_0, x_1, x_2 x_plot = np.linspace(0.8, 2.1, 400) fig, ax = plt.subplots(figsize=(8, 6)) ax.plot(x_plot, f(x_plot), color=\u0026#34;C0\u0026#34;, label=r\u0026#34;$f(x) = x^2 - 2$\u0026#34;, zorder=2) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) for i in range(len(xs_plot) - 1): x_curr = xs_plot[i] x_new = xs_plot[i + 1] # tangent line at x_curr, drawn until it crosses y=0 (which is x_new) tangent_x = np.array([x_curr, x_new]) tangent_y = np.array([f(x_curr), 0.0]) ax.plot(tangent_x, tangent_y, color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, linewidth=1.5, zorder=1) ax.scatter([x_curr], [f(x_curr)], color=\u0026#34;C1\u0026#34;, zorder=3) ax.annotate(f\u0026#34;$x_{i}$\u0026#34;, (x_curr, f(x_curr)), textcoords=\u0026#34;offset points\u0026#34;, xytext=(6, 8)) ax.axvline(np.sqrt(2), color=\u0026#34;C2\u0026#34;, linestyle=\u0026#34;:\u0026#34;, label=r\u0026#34;$\\sqrt{2}$ (exact root)\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;f(x)\u0026#34;) ax.set_title(r\u0026#34;Newton step by step: successive tangents converging to $\\sqrt{2}$\u0026#34;) Newton step by step: each tangent line points to the next estimate, closer and closer to √2. The official SciPy version does exactly this, only faster and with more robust stopping criteria:\nfrom scipy import optimize def f(x): return x**2 - 2 def fp(x): return 2 * x print(optimize.newton(f, x0=1.0, fprime=fp)) 1.4142135623730951 When Newton Fails # Newton\u0026rsquo;s speed has a price: without the guarantee of a bracket, it can diverge. And we do not even need a contrived example, because the following case is a classic of any numerical analysis textbook: \\(f(x) = x^3 - 2x + 2\\), starting from \\(x_0 = 0\\).\nfrom scipy import optimize def f(x): return x**3 - 2*x + 2 def fp(x): return 3*x**2 - 2 optimize.newton(f, x0=0.0, fprime=fp) RuntimeError: Failed to converge after 50 iterations, value is 0.0. To see exactly what happens, it is worth working out the first two steps by hand, using the same formula \\(x_{n+1} = x_n - f(x_n)/f^{\\prime}(x_n)\\) that we just derived:\nAt \\(x_0 = 0\\): \\(f(0) = 2\\) and \\(f^{\\prime}(0) = -2\\), so \\(x_1 = 0 - \\dfrac{2}{-2} = 1\\). At \\(x_1 = 1\\): \\(f(1) = 1\\) and \\(f^{\\prime}(1) = 1\\), so \\(x_2 = 1 - \\dfrac{1}{1} = 0\\), back, exactly, to \\(x_0\\). The method returns exactly to the start, with no approximation: it is a perfect cycle of period 2. From here on, the method jumps forever between \\(0\\) and \\(1\\), never getting close to the real root (which exists, and is unique, around \\(x \\approx -1.77\\)). The following figure draws this back and forth: the two dashed tangent lines, which cross instead of converging, and the dotted lines that close the cycle (down the tangent to the axis, back up to the curve, repeat).\nimport matplotlib.pyplot as plt import numpy as np def f(x): return x**3 - 2*x + 2 real_root = -1.7692923542386314 x_vals = np.linspace(-2.5, 2.5, 500) fig, ax = plt.subplots(figsize=(8, 5.5)) ax.plot(x_vals, f(x_vals), color=\u0026#34;C0\u0026#34;, label=r\u0026#34;$f(x) = x^3 - 2x + 2$\u0026#34;, zorder=2) ax.axhline(0, color=\u0026#34;gray\u0026#34;, linewidth=1, zorder=0) # tangent at x=0 leads to x=1; tangent at x=1 leads back to x=0 for x_curr, x_new in [(0.0, 1.0), (1.0, 0.0)]: ax.plot([x_curr, x_new], [f(x_curr), 0.0], color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;--\u0026#34;, linewidth=1.6, zorder=3) ax.plot([x_new, x_new], [0.0, f(x_new)], color=\u0026#34;C1\u0026#34;, linestyle=\u0026#34;:\u0026#34;, linewidth=1.2, alpha=0.7, zorder=1) ax.scatter([0.0, 1.0], [f(0.0), f(1.0)], color=\u0026#34;C1\u0026#34;, s=60, zorder=4, label=\u0026#34;cycle tangents\u0026#34;) ax.scatter([real_root], [0.0], color=\u0026#34;C2\u0026#34;, s=90, marker=\u0026#34;*\u0026#34;, zorder=4, label=f\u0026#34;real root (≈ {real_root:.2f})\u0026#34;) ax.set_ylim(-2.5, 5.5) ax.legend(loc=\u0026#34;upper left\u0026#34;) ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;f(x)\u0026#34;) ax.set_title(\u0026#34;Newton in a cycle: the two tangents that alternate forever\u0026#34;) The two tangents form an X: the one at x₀=0 points to x₁=1, the one at x₁=1 points back to x₀=0. The method stays stuck in this pair forever, while the real root (star) is on the other side of the plot. With a better initial guess, the same method converges without obstacles:\nfrom scipy import optimize def f(x): return x**3 - 2*x + 2 def fp(x): return 3*x**2 - 2 print(optimize.newton(f, x0=-2.0, fprime=fp)) -1.7692923542386314 Sweeping Thousands of Initial Guesses # It might seem that \\(x_0 = 0\\) was just an unlucky guess, a singular point that can be avoided with a little care. That is not the case. We can sweep thousands of initial guesses, run Newton from each one, and classify: did it converge to the real root, or not?\nimport numpy as np def f(x): return x**3 - 2*x + 2 def fp(x): return 3*x**2 - 2 real_root = -1.7692923542386314 def newton_destination(x0, n_iter=200): \u0026#34;\u0026#34;\u0026#34;Run Newton from x0 and return the final point (or nan if it escaped).\u0026#34;\u0026#34;\u0026#34; x = float(x0) for _ in range(n_iter): derivative = fp(x) if abs(derivative) \u0026lt; 1e-14: return np.nan x = x - f(x) / derivative if not np.isfinite(x) or abs(x) \u0026gt; 1e8: return np.nan return x x0_grid = np.linspace(-3, 3, 6001) converged = np.array([abs(newton_destination(x0) - real_root) \u0026lt; 1e-6 for x0 in x0_grid]) print(f\u0026#34;fraction of guesses that converge: {converged.mean():.1%}\u0026#34;) fraction of guesses that converge: 80.2% Almost one in every five initial guesses in this range fails, and their distribution is anything but intuitive, as the following figure shows, with the \\(x_0\\) that converge painted green and those that do not painted red:\nNewton basins of attraction: each position on the strip is an initial guess. Green converges to the real root; red does not. Between the dotted lines (where f\u0026rsquo;(x)=0), the boundary between the two regimes becomes an increasingly fine alternation. Far from the center, everything is green: Newton behaves. But between the two dotted lines, which mark where \\(f^{\\prime}(x) = 3x^2 - 2 = 0\\), that is, \\(x = \\pm\\sqrt{2/3} \\approx \\pm 0.816\\), things fall apart. There is a wide red band around \\(x_0 = 0\\) (our cycle, which drags a whole neighborhood along with it) and, around it, green and red stripes that alternate ever finer as they approach the points where the derivative vanishes. In this interval, this alternation never \u0026ldquo;ends\u0026rdquo;: no matter how far you zoom in, stripes keep appearing, because the boundary between converging and not converging is fractal.\nThat is why \u0026ldquo;choose a better initial guess\u0026rdquo; is harder advice to follow than it seems: over a good part of the domain, neighboring guesses can have completely different destinations.\nThe lesson of Newton Newton is not \u0026ldquo;no good\u0026rdquo;: it is sensitive to the initial guess in a way that brentq simply is not (as long as the bracket is valid). Treat newton as the fast tool for when you already have a good idea of where the root is; treat brentq as the default tool, more tolerant of initial ignorance.\nComparing Orders of Convergence # The \u0026ldquo;quadratic convergence\u0026rdquo; (Newton) and the robustness guarantee of bisection make more sense side by side, with numbers. Let\u0026rsquo;s solve the same \\(f(x) = x^2 - 2\\) with both methods and track the error at each iteration, on a logarithmic scale. To do this, we reimplement bisection by hand instead of calling scipy.optimize.bisect directly, the only way to store the value at each iteration for the plot, since the ready-made function only returns the final result.\nA detail about bisection The pointwise error of the midpoint, \\(|x_n - \\sqrt{2}|\\), does not decrease monotonically: the midpoint may \u0026ldquo;jump\u0026rdquo; to the other side of the root at each step. This is normal, but it produces a jagged and confusing plot. What bisection really guarantees, always, is to halve the interval width at each iteration, and this width is a reliable upper bound on the error. That is the quantity that appears in the plot below, in place of the raw pointwise error.\nimport matplotlib.pyplot as plt import numpy as np def f(x): return x**2 - 2 def fp(x): return 2 * x def newton_manual(f, fp, x0, n_iter): xs = [x0] for _ in range(n_iter): x_curr = xs[-1] xs.append(x_curr - f(x_curr) / fp(x_curr)) return xs def bisection_with_history(f, a, b, n_iter): xs, widths = [], [] for _ in range(n_iter): m = (a + b) / 2 xs.append(m) widths.append(b - a) if np.sign(f(a)) == np.sign(f(m)): a = m else: b = m return xs, widths exact_root = np.sqrt(2) n_iter = 12 xs_bisection, widths_bisection = bisection_with_history(f, a=1.0, b=2.0, n_iter=n_iter) xs_newton_conv = newton_manual(f, fp, x0=1.0, n_iter=n_iter)[1:] newton_error = [abs(x - exact_root) for x in xs_newton_conv] fig, ax = plt.subplots(figsize=(8, 5)) ax.semilogy(range(1, len(widths_bisection) + 1), widths_bisection, \u0026#34;o-\u0026#34;, color=\u0026#34;C0\u0026#34;, label=\u0026#34;bisection (interval width, linear)\u0026#34;) ax.semilogy(range(1, len(newton_error) + 1), newton_error, \u0026#34;o-\u0026#34;, color=\u0026#34;C1\u0026#34;, label=\u0026#34;Newton (error, quadratic)\u0026#34;) ax.axhline(1e-15, color=\u0026#34;gray\u0026#34;, linestyle=\u0026#34;:\u0026#34;, linewidth=1, label=\u0026#34;machine precision\u0026#34;) ax.set_xlabel(\u0026#34;iteration\u0026#34;) ax.set_ylabel(\u0026#34;error (or upper bound on the error)\u0026#34;) ax.set_title(\u0026#34;Order of convergence: bisection vs. Newton\u0026#34;) ax.legend() Order of convergence: bisection reduces the error by a constant factor at each step (a straight line on a log scale); Newton doubles the correct digits at each step (a dizzying drop). The visual difference is glaring: bisection is a well-behaved straight line on a log scale (it loses a constant amount of error at each step, that is, linear convergence); Newton plunges almost vertically after a few iterations (quadratic convergence). And brentq, which combines both worlds, solves the same problem in very few iterations:\nfrom scipy import optimize def f(x): return x**2 - 2 _, info = optimize.brentq(f, 1.0, 2.0, full_output=True) print(\u0026#34;brentq converged in\u0026#34;, info.iterations, \u0026#34;iterations,\u0026#34;, info.function_calls, \u0026#34;function evaluations\u0026#34;) brentq converged in 7 iterations, 8 function evaluations Fast like Newton, without requiring a derivative and without the risk of divergence, in most practical cases.\nSystems of Nonlinear Equations: fsolve and root # Everything so far solved one equation in one variable. And when there is more than one unknown and the equations are not linear, a situation in which sympy.solve has no guarantee of finding (or cannot always find) a closed form? SciPy solves it with scipy.optimize.fsolve.\nConsider the intersection between a circle, \\(x^2+y^2=4\\), and a parabola, \\(y=x^2\\), with two solutions by symmetry in \\(x\\).\nfrom scipy import optimize def system(vars_): x, y = vars_ return [x**2 + y**2 - 4, y - x**2] sol_a = optimize.fsolve(system, [1, 1]) sol_b = optimize.fsolve(system, [-1, 1]) print(\u0026#34;from (1, 1):\u0026#34;, sol_a) print(\u0026#34;from (-1, 1):\u0026#34;, sol_b) from (1, 1): [1.24962107 1.56155281] from (-1, 1): [-1.24962107 1.56155281] The plot below shows the two curves and the two solutions found.\nimport matplotlib.pyplot as plt import numpy as np from scipy import optimize def system(vars_): x, y = vars_ return [x**2 + y**2 - 4, y - x**2] sol_a = optimize.fsolve(system, [1, 1]) sol_b = optimize.fsolve(system, [-1, 1]) theta = np.linspace(0, 2 * np.pi, 300) x_circ, y_circ = 2 * np.cos(theta), 2 * np.sin(theta) x_par = np.linspace(-2, 2, 300) y_par = x_par**2 fig, ax = plt.subplots(figsize=(7, 7)) ax.plot(x_circ, y_circ, color=\u0026#34;C0\u0026#34;, label=r\u0026#34;$x^2+y^2=4$\u0026#34;) ax.plot(x_par, y_par, color=\u0026#34;C1\u0026#34;, label=r\u0026#34;$y=x^2$\u0026#34;) ax.scatter(*sol_a, color=\u0026#34;C2\u0026#34;, zorder=3) ax.scatter(*sol_b, color=\u0026#34;C2\u0026#34;, zorder=3, label=\u0026#34;solutions (fsolve)\u0026#34;) ax.set_aspect(\u0026#34;equal\u0026#34;) ax.legend() ax.set_xlabel(\u0026#34;x\u0026#34;) ax.set_ylabel(\u0026#34;y\u0026#34;) ax.set_title(\u0026#34;Nonlinear system: circle × parabola\u0026#34;) Nonlinear system: circle × parabola, with the two solutions found by fsolve from different initial guesses. Note that we had to vary the initial guess to find each solution, unlike sympy.solve, which would return both at once when it manages to solve symbolically. fsolve finds one solution per call, the one closest to the guess provided.\nWhat if the Initial Guess Is Bad? # Just as brentq requires a valid bracket and Newton can diverge with a bad guess, fsolve also has no guarantee of convergence and, unlike brentq, does not warn you with an error by default. To investigate, we use full_output=True, which makes fsolve return, besides the solution, a status code called ier, a simple integer that works as a flag: 1 means success; any other value indicates that something went wrong (the specific reason is described separately, in the msg message). This pattern of returning an integer as a \u0026ldquo;verdict\u0026rdquo; comes from the numerical routines of the MINPACK package (originally written in Fortran) that fsolve uses under the hood.\nimport numpy as np from scipy import optimize def system(vars_): x, y = vars_ return [x**2 + y**2 - 4, y - x**2] solution, info, ier, msg = optimize.fsolve(system, [0, 0], full_output=True) print(\u0026#34;returned solution:\u0026#34;, solution) print(\u0026#34;converged (ier == 1)?\u0026#34;, ier == 1) print(\u0026#34;solver message:\u0026#34;, msg) print(\u0026#34;residual at the returned point:\u0026#34;, np.array(system(solution))) returned solution: [0. 0.] converged (ier == 1)? False solver message: The iteration is not making good progress, as measured by the improvement from the last ten iterations. residual at the returned point: [-4. 0.] fsolve did not even move: it returned the guess itself as the \u0026ldquo;solution,\u0026rdquo; with residual \\([-4, 0]\\), clearly wrong. The reason is exactly the same one that makes Newton\u0026rsquo;s method stall when the derivative is zero, only generalized to systems: in place of the derivative \\(f^{\\prime}(x)\\), we have the matrix of partial derivatives of all the equations with respect to all the unknowns, the Jacobian. For our system,\n$$ J(x, y) = \\begin{pmatrix} \\partial_x (x^2+y^2-4) \u0026 \\partial_y (x^2+y^2-4) \\\\ \\partial_x (y - x^2) \u0026 \\partial_y (y - x^2) \\end{pmatrix} = \\begin{pmatrix} 2x \u0026 2y \\\\ -2x \u0026 1 \\end{pmatrix} $$Evaluating at the guess \\((0,0)\\):\n$$ J(0, 0) = \\begin{pmatrix} 0 \u0026 0 \\\\ 0 \u0026 1 \\end{pmatrix} $$The entire first row vanishes. This means that, in the neighborhood of \\((0,0)\\), the circle equation does not \u0026ldquo;respond\u0026rdquo; to any change in \\(x\\) or in \\(y\\): to first order, it is insensitive to both unknowns at once. The determinant is zero, the Jacobian is singular, and the generalized Newton step (which needs to solve a linear system with this matrix) has nowhere to go. Hence fsolve returns the guess itself.\nAlways check ier and the residual By default, fsolve returns only the solution array; without full_output=True, the error above would pass silently. In any serious use, it is worth checking ier or, at the very least, substituting the \u0026ldquo;solution\u0026rdquo; back into the system and confirming that the residual is close to zero.\nfsolve Is a Wrapper: Meet root # fsolve has existed in SciPy for decades and shares the same core (MINPACK) with a newer and more uniform interface: scipy.optimize.root. The advantage of root is that it always returns an OptimizeResult object, with .x (the solution) and .success (a boolean) directly, without having to remember to pass full_output=True to find out whether it converged.\nfrom scipy import optimize def system(vars_): x, y = vars_ return [x**2 + y**2 - 4, y - x**2] result = optimize.root(system, [1, 1]) print(\u0026#34;solution:\u0026#34;, result.x) print(\u0026#34;converged?\u0026#34;, result.success) solution: [1.24962107 1.56155281] converged? True And, of course, root suffers from exactly the same problem with the guess \\((0, 0)\\), because internally root\u0026rsquo;s default method ('hybr') is the same algorithm that fsolve uses:\nfrom scipy import optimize def system(vars_): x, y = vars_ return [x**2 + y**2 - 4, y - x**2] bad_result = optimize.root(system, [0, 0]) print(\u0026#34;solution:\u0026#34;, bad_result.x) print(\u0026#34;converged?\u0026#34;, bad_result.success) print(\u0026#34;message:\u0026#34;, bad_result.message) solution: [0. 0.] converged? False message: The iteration is not making good progress, as measured by the improvement from the last ten iterations. Same result as before, but with a direct success check, without full_output. root also accepts other algorithms via method ('broyden1', 'krylov', among others) for larger or harder systems; for the systems in this article, the default already does well. The recommendation stands: in new code, prefer root to fsolve just for the clarity of the interface.\nSummary Table of the Methods # The table below summarizes when to use each method seen in this article.\nMethod Needs a bracket? Needs a derivative? Guaranteed convergence? Order of convergence When to use bisect Yes (sign change) No Yes, if the bracket is valid Linear Robustness above all; speed does not matter brentq Yes (sign change) No Yes, if the bracket is valid Superlinear, fast in practice Default choice for one variable newton (with fprime) No Yes No, depends on the initial guess Quadratic Derivative available and a good initial guess newton (without fprime) No No (secant) No, depends on the initial guess Superlinear No derivative, but a reasonable guess fsolve / root No Optional (Jacobian) No, depends on the initial guess Varies with the inner method Systems with more than one equation/variable Rule of thumb: for a single variable, start with brentq, which does not require a derivative and is reliable. Use newton when you already have the derivative and need more speed. For systems, fsolve/root, but always validate the residual, because there is no guarantee of convergence.\nNext Steps # All the methods in this article were tested on didactic problems: equations chosen to illustrate a specific behavior (a bracket, a divergence, a slow convergence), not to solve a real problem from end to end. What remains is to apply these same tools to equations that come from a real physical, chemical, or engineering problem, with the units, the interpretations, and the pitfalls that only appear outside the didactic lab.\nIn fact, the question that opened this article is still pending: what, after all, is the position of a planet on an elliptical orbit given its mean anomaly and eccentricity? Now we have the tools to answer: newton, with the exact derivative of Kepler\u0026rsquo;s equation, converges rapidly to the eccentric anomaly \\(E\\). In the next article, we solve this equation for real, visualize the resulting orbit, and apply the same methods to the range of a projectile under air resistance, another physics classic with no closed-form solution.\nAfterward, in From Chemistry to Engineering, we will see three real equations from chemistry and engineering (the molar volume of a real gas, the pH of a weak acid, and the Colebrook friction factor), all solved with the same tools presented here.\n","date":"2026-09-02","externalUrl":null,"permalink":"/2026/09/numerical-equations-scipy/","section":"Blog","summary":"In Kepler’s equation, the unknown appears both inside and outside a sine, and no algebra isolates it. For cases like this, SciPy approximates the root. We will see bisection and brentq, Newton’s method and why it can fail, and nonlinear systems with fsolve and root.","title":"Solving Equations Numerically with SciPy","type":"posts"},{"content":" Many of the older articles on this site about SymPy solve problems symbolically: an equation gets a closed formula, a derivative comes out exact, an infinite series converges to a clean expression. It is elegant mathematics and, away from the blackboard, incomplete. The vast majority of the equations, integrals, and differential equations that show up in real problems in physics, chemistry, and engineering have no closed-form solution, and much of the data we need to differentiate has no formula behind it at all, just a table of measurements. Newton\u0026rsquo;s method, a finite difference, a numerical quadrature: these are the tools that solve what exact algebra can\u0026rsquo;t reach. This series shows how.\nWhy Does This Series Matter? # As soon as an equation gains a nonlinear term, a realistic boundary condition, or more than one coupled variable, the chance of an exact formula existing collapses, and the numerical route becomes the only practical way out. NumPy and SciPy are the standard pair for scientific computing in Python, used in industry, research, and teaching. Knowing when (and why) to swap sympy.solve for scipy.optimize.brentq, or an exact integral for a numerical grid, is a practical skill that few courses teach directly. Each article starts from a real problem: planetary orbits, solution pH, structural beams, nuclear magnetic resonance. The series avoids artificial textbook equations whenever a genuine example serves just as well. The Series Map # Each block of the series stems from a concrete limitation of symbolic algebra. The diagram below shows, for each one, the numerical tool that solves it and where it is presented:\n\u0026#x2197; flowchart LR A1[\"Equation with no\nclosed form\"] --\u003e B1[\"scipy.optimize\nbrentq · newton · root\"] --\u003e C1[\"Block 1\n3 articles\"] A2[\"Data with no\nformula behind it\"] --\u003e B2[\"numpy.polynomial\nfit · Chebyshev\"] --\u003e C2[\"Block 2\n2 articles\"] A3[\"Derivative and integral\nthat don't close\"] --\u003e B3[\"gradient · quad\nsolve_ivp\"] --\u003e C3[\"Block 3\n3 articles\"] A4[\"Minimum subject\nto constraints\"] --\u003e B4[\"minimize · SLSQP\ncurve_fit\"] --\u003e C4[\"Block 4\n3 articles\"] A5[\"Signal measured\nover time\"] --\u003e B5[\"numpy.fft\nrfft · ifft\"] --\u003e C5[\"Block 5\n2 articles\"] What You Will Learn # The series has 13 articles, organized into the five thematic blocks above, each one solving a type of problem that SymPy can\u0026rsquo;t close:\nBlock 1: Solving Equations With No Closed Form # Three articles to build the foundations of root finding with scipy.optimize and apply them to real problems in physics, chemistry, and engineering, from Kepler\u0026rsquo;s equation to the Colebrook friction factor.\n→ Solving Equations Numerically with SciPy → Orbits and Trajectories: Numerical Equations in Physics → From Chemistry to Engineering: Numerical Equations in Real Systems\nBlock 2: Polynomials: Manipulating and Fitting # Two articles on numpy.polynomial: first as a known object (roots, derivative, integral), then as the result of fitting noisy experimental data, including Runge\u0026rsquo;s phenomenon and the ill-conditioning of Wilkinson\u0026rsquo;s polynomial.\n→ Polynomials with NumPy → Curve Fitting with NumPy\nBlock 3: Numerical Calculus: Differentiating, Integrating, Solving # Three articles on calculus when there is no formula: differentiating data that has no expression behind it, integrating functions that have no elementary antiderivative, and solving differential equations, from the nonlinear pendulum to stiff systems, with scipy.integrate.solve_ivp.\n→ Numerical Derivatives with SciPy → Numerical Integration with SciPy → Differential Equations with SciPy\nBlock 4: Optimization # Three articles on scipy.optimize: the fundamentals (constraints, bounds, local × global minima), four real applications, from a chemical reactor to the production mix of a plant, and, closing the block, fitting nonlinear models to experimental data, which is optimization disguised as curve fitting, with the parameter uncertainties that come along with it.\n→ Optimization with SciPy → Applied Optimization with SciPy → Nonlinear Fitting and Uncertainties with SciPy\nBlock 5: Fourier and FFT # Two articles close the series: from the symbolic Fourier series to the fast Fourier transform in practice, with its two classic artifacts (aliasing and spectral leakage), through to real applications in nuclear magnetic resonance, mechanical vibration, and audio.\n→ From the Fourier Series to the FFT with NumPy → Applied FFT: From Magnetic Resonance to Audio\nWho Is This Series For? # The series is for those who already have some familiarity with Python and calculus-level mathematics (derivatives, integrals, differential equations at an introductory level) and want to learn how to computationally attack the problems that symbolic manipulation doesn\u0026rsquo;t solve.\nYou don\u0026rsquo;t need to have used NumPy or SciPy before: each tool is introduced in the context of the problem it solves, with a direct comparison against the symbolic route whenever possible.\nPrerequisites # Several articles revisit examples and results already published in this site\u0026rsquo;s SymPy series. It is worth knowing, although not strictly necessary:\nIntroduction to SymPy: first contact with the computer algebra package Derivatives with SymPy, Optimization with SymPy, and Infinite Series with SymPy: revisited directly at several points in this series If you don\u0026rsquo;t know SymPy yet, that\u0026rsquo;s fine: each revisit comes with its context restated, without requiring you to read the original article.\nQuick Reference Table # # Article What It Solves 1 Numerical Equations Root criterion, brentq/newton/fsolve 2 Orbits and Trajectories Kepler\u0026rsquo;s equation, projectile with drag 3 Chemistry and Engineering pH, van der Waals, friction factor 4 Polynomials Polynomial, roots, ill-conditioning 5 Curve Fitting Regression, Runge\u0026rsquo;s phenomenon, splines 6 Numerical Derivatives Finite differences, numpy.gradient 7 Numerical Integration quad, simpson, functions with no antiderivative 8 Differential Equations solve_ivp, nonlinear pendulum, stiffness 9 Optimization minimize, SLSQP, constraints 10 Applied Optimization Reactor, projectile, beam, production mix 11 Nonlinear Fitting curve_fit, parameter uncertainty 12 Fourier and FFT fft, aliasing, Gibbs phenomenon 13 Applied FFT NMR, vibration, audio Ready to start? The first article begins with a simple problem, an equation that no formula closes, to build, from scratch, the criterion that decides when it is worth giving up the search for an exact solution: Solving Equations Numerically with SciPy.\n","date":"2026-09-01","externalUrl":null,"permalink":"/2026/09/from-symbol-to-number/","section":"Blog","summary":"Most equations, integrals, and differential equations in real problems have no closed-form solution, and SymPy can’t solve them. This 13-article series shows how NumPy and SciPy tackle these problems: roots, polynomials, numerical calculus, optimization, and FFT.","title":"From Symbol to Number: SciPy and NumPy Beyond SymPy","type":"posts"},{"content":"Uma introdução completa à Teoria dos Grafos: das Pontes de Königsberg aos grafos direcionados. Aprenda a matemática que está por trás das redes, algoritmos e conexões do mundo digital — grau, isomorfismo, caminhos, árvores, planaridade e mais.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-08-27","externalUrl":null,"permalink":"/series/a-matematica-das-conexoes/","section":"Séries","summary":"","title":"A Matemática das Conexões","type":"series"},{"content":" So far, we have dealt with graphs whose relations are symmetric: if \\(A\\) is connected to \\(B\\), then \\(B\\) is also connected to \\(A\\). But many real-world relations are asymmetric.\nA one-way street goes from \\(A\\) to \\(B\\), but not from \\(B\\) to \\(A\\). A hyperlink leads from one page to another, but the other page does not necessarily point back. A task \\(A\\) must be completed before \\(B\\) starts, and not the other way around.\nTo model these situations, we need directed graphs.\nWhy Do Digraphs Matter? # Whenever a relation between two objects is not automatically reciprocal, an ordinary graph is not enough: we also have to record its direction. Some examples:\nNavigation and maps: one-way streets and air routes cannot always be traveled in both directions The web and search engines: hyperlinks point from one page to another, and Google\u0026rsquo;s PageRank (already seen in Graphs: A Language for Modeling Connections) is built on this structure Social networks: following someone on Instagram or X does not imply being followed back Compilers and package managers: dependencies between modules can only be resolved in a specific order Task scheduling: a project with steps that must happen in sequence To make the idea concrete before the formal definition, consider the following scenario.\nExample: a one-way city block Problem: Consider a block in Copacabana, in Rio de Janeiro, bounded by four streets (Avenida Atlântica, Rua Figueiredo Magalhães, Avenida Nossa Senhora de Copacabana, and Rua Santa Clara) with Rua Domingos Ferreira cutting the block in half. Model this block first as an ordinary graph \\(G\\) and then as a digraph \\(D\\), recording the traffic direction of each street.\nSolution: Each intersection is a vertex, identified by the two streets that cross there: \\(AF\\) (Atlântica × Figueiredo), \\(AS\\) (Atlântica × Santa Clara), \\(DF\\) (Domingos Ferreira × Figueiredo), \\(DS\\) (Domingos Ferreira × Santa Clara), \\(NF\\) (N. S. Copacabana × Figueiredo), and \\(NS\\) (N. S. Copacabana × Santa Clara). Each stretch of street between two neighboring intersections is an edge.\nIgnoring the traffic direction, this is already an ordinary graph \\(G\\), with edges $$(AF,AS), (AF,DF), (DF,DS), (DF,NF), (DS,AS), (NS,DS), (NF,NS)$$But the streets have traffic directions: Av. Atlântica is two-way; Rua Figueiredo Magalhães is one-way in the direction \\(AF \\to DF \\to NF\\); Rua Santa Clara is one-way in the opposite direction, \\(NS \\to DS\\) and \\(DS \\to AS\\); Rua Domingos Ferreira flows from \\(DF\\) to \\(DS\\); and Av. Nossa Senhora de Copacabana flows from \\(NS\\) to \\(NF\\). Recording each direction as an arc, we get the digraph $$D: \\{(AF,AS), (AS,AF), (AF,DF), (DF,NF), (NS,DS), (DS,AS), (DF,DS), (NS,NF)\\}$$ The digraph is the same block, now with the information about where traffic can go on each street. The figure below shows the two versions side by side:\nThe block as a graph G and as a digraph D What Is a Digraph? # Definition (Digraph, or Directed Graph) A digraph \\(D = (V, E)\\) is a pair where:\n\\(V\\) is a finite nonempty set of vertices \\(E\\) is a set of arcs: ordered pairs of distinct vertices For an arc \\((v, w) \\in E(D)\\):\n\\(v\\) is the tail (starting point) \\(w\\) is the head (end point) We say the arc diverges from \\(v\\) and converges to \\(w\\) In the geometric representation, we draw an arrow from \\(v\\) to \\(w\\) Fundamental difference In a (undirected) graph, the edge \\(\\{u,v\\} = \\{v,u\\}\\), with no distinction.\nIn a digraph, the arc \\((u,v) \\neq (v,u)\\): they are different arcs! Both can exist at the same time.\nTail and head of an arc; (v,w) and (w,v) are different arcs Concrete example Problem: For the digraph \\(D\\) with \\(V = \\{a, b, c, d, e\\}\\) and \\(E = \\{(a,b),(b,c),(a,e),(b,e),(e,b),(e,c),(d,c),(d,e)\\}\\), describe the adjacency relations of each vertex.\nSolution:\nDigraph D: V={a,b,c,d,e} \\(a\\): points to \\(b\\) and \\(e\\) (but nobody points to \\(a\\)) \\(b\\) and \\(e\\) have arcs in both directions between them: \\((b,e)\\) and \\((e,b)\\) \\(c\\): receives arcs from \\(b\\), \\(e\\), and \\(d\\), but points to nobody Underlying Graph and Orientation # Underlying Graph The underlying graph of a digraph \\(D\\) is obtained by removing the directions of the arcs (turning them into undirected edges) and eliminating the resulting parallel edges.\nExample: building the underlying graph Problem: Build the underlying graph of the digraph \\(D\\) from the previous example (\\(V=\\{a,b,c,d,e\\}\\), arcs \\(\\{(a,b),(b,c),(a,e),(b,e),(e,b),(e,c),(d,c),(d,e)\\}\\)).\nSolution: The underlying graph is built in two steps:\nRemove the directions: each arc becomes an ordinary edge. Since \\((b,e)\\) and \\((e,b)\\) were two distinct arcs between the same vertices, they become two parallel edges between \\(b\\) and \\(e\\). Remove the parallel edges: the two edges between \\(b\\) and \\(e\\) collapse into one. The result is the underlying graph \\(G\\), with edges \\((a,b), (a,e), (b,c), (b,e), (e,c), (c,d), (d,e)\\), 7 edges, one fewer than the 8 original arcs of \\(D\\). The figure below shows the two steps:\nBuilding the underlying graph of D in two steps Orientation An orientation of a graph \\(G\\) is a digraph \\(\\vec{G}\\) obtained by specifying a direction for each edge of \\(G\\). The orientation of a graph is not unique: the same \\(G\\) admits several different orientations.\nExample: two different orientations of the same graph Problem: Find two different orientations of the underlying graph \\(G\\) from the previous example.\nSolution: Two possibilities:\n\\(\\vec{G}\\): \\((a,b), (a,e), (b,c), (e,b), (c,e), (c,d), (e,d)\\) \\((\\vec{G})^{\\prime}\\): the opposite orientation of each edge, \\((b,a), (e,a), (c,b), (b,e), (e,c), (d,c), (d,e)\\) Both are valid digraphs over the same underlying graph \\(G\\), but with arcs pointing in different directions, which is why the orientation of a graph is not unique, as the figure shows:\nG admits more than one orientation In-Degree and Out-Degree # In digraphs, each vertex has two degrees:\nIn-Degree and Out-Degree For a vertex \\(v\\) in a digraph \\(D\\):\nIn-degree: \\(d^-(v) = |\\{(u,v) \\in E(D)\\}|\\), the number of arcs arriving at \\(v\\) Out-degree: \\(d^+(v) = |\\{(v,u) \\in E(D)\\}|\\), the number of arcs leaving \\(v\\) Special vertices:\nSource: \\(d^-(v) = 0\\), that is, no arc arrives at \\(v\\) Sink: \\(d^+(v) = 0\\), that is, no arc leaves \\(v\\) Handshaking-Like Formula # Theorem $$\\sum_{v \\in V} d^-(v) = \\sum_{v \\in V} d^+(v) = |E(D)|$$Proof: Each arc \\((u,v)\\) contributes 1 to \\(d^+(u)\\) and 1 to \\(d^-(v)\\). Summing over all arcs, each sum counts each arc exactly once. Therefore, \\(\\sum d^-(v) = \\sum d^+(v) = |E(D)|\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExample: degrees in the digraph above Problem: Compute the in-degree and the out-degree of each vertex of the digraph \\(D\\) from the example above (\\(V = \\{a,b,c,d,e\\}\\)), and identify sources and sinks.\nSolution:\nVertex \\(d^+(v)\\) \\(d^-(v)\\) \\(a\\) 2 0 \\(b\\) 2 2 \\(c\\) 0 3 \\(d\\) 2 0 \\(e\\) 2 3 \\(a\\) and \\(d\\) are sources; \\(c\\) is a sink.\nSum of the out-degrees: \\(2+2+0+2+2 = 8 = |E|\\). ✓\nChecking with code Counting arcs by hand works for small digraphs, but networkx already provides the in-degree and the out-degree as methods of the digraph itself: D.in_degree(v) and D.out_degree(v). The code below goes through each vertex, computes both degrees, and marks which one is a source (\\(d^-=0\\)) or a sink (\\(d^+=0\\)):\nimport networkx as nx D = nx.DiGraph([(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;e\u0026#34;)]) for v in sorted(D.nodes()): d_plus, d_minus = D.out_degree(v), D.in_degree(v) role = \u0026#34; (source)\u0026#34; if d_minus == 0 else \u0026#34; (sink)\u0026#34; if d_plus == 0 else \u0026#34;\u0026#34; print(f\u0026#34;{v}: d+={d_plus}, d-={d_minus}{role}\u0026#34;) a: d+=2, d-=0 (source) b: d+=2, d-=2 c: d+=0, d-=3 (sink) d: d+=2, d-=0 (source) e: d+=2, d-=3 The result matches the table computed by hand exactly.\nPaths and Cycles in Digraphs # The definitions of walk, trail, path, and cycle, already seen in Paths, Cycles, and Connectivity, still hold, but now with an important restriction: we must follow the directions of the arcs.\nDirected Walk A directed walk in a digraph \\(D\\) is a sequence of vertices \\(P = v_0, v_1, \\ldots, v_k\\) such that \\((v_i, v_{i+1}) \\in E(D)\\) for \\(0 \\leq i \\leq k-1\\), that is, each step of the sequence follows an arc of \\(D\\) in the indicated direction. Vertices and arcs can repeat.\nDirected Trail A directed trail is a directed walk in which no arc repeats (but vertices can repeat).\nDirected Path A directed path is a directed walk in which no vertex repeats (and therefore no arc repeats either).\nExample: walk, trail, and path in the same digraph Problem: Consider the digraph with \\(V = \\{a,b,c,d,e,f,g,h\\}\\) and arcs \\(E = \\{(a,b),(b,c),(b,e),(c,e),(c,g),(e,a),(e,d),(d,a),(d,c),(d,f),(f,d),(d,h)\\}\\), shown in the figure below. Classify the sequences \\(P_1 = a,b,c,e,a,b,c,g\\), \\(P_2 = a,b,e,d,f,d,c,g\\), and \\(P_3 = a,b,c,g\\).\nDigraph used in the examples of walk, trail, path, and reachability Solution:\n\\(P_1\\) is a directed walk from \\(a\\) to \\(g\\): the arcs \\((a,b)\\) and \\((b,c)\\) are traversed twice, so it is not a trail. \\(P_2\\) is a directed trail from \\(a\\) to \\(g\\): no arc repeats, but the vertex \\(d\\) appears twice, so it is not a path. \\(P_3\\) is a directed path from \\(a\\) to \\(g\\): no vertex repeats. In addition, \\(C_1 = a,b,c,e,d,a\\) is a directed cycle, and \\(C_2 = d,f,d\\) is a directed cycle of length 2, possible because \\((d,f)\\) and \\((f,d)\\) are both arcs of \\(D\\).\nNote The cycle \\(C_2 = d,f,d\\) in the example above illustrates a fact exclusive to digraphs: if \\((u,v)\\) and \\((v,u)\\) are both arcs, then \\(u \\to v \\to u\\) is a cycle of length 2. In simple (undirected) graphs, this does not happen, since the same edge cannot be traversed twice in a cycle.\nReachability: we say that \\(v\\) reaches \\(u\\) if there is a directed path from \\(v\\) to \\(u\\).\nExample: reachability Problem: In the digraph of the previous figure, which vertices does \\(a\\) reach? Is there any vertex that does not reach \\(a\\)?\nSolution: \\(a\\) reaches \\(g\\) (via \\(a,b,c,g\\)), reaches \\(f\\) (via \\(a,b,e,d,f\\)), and reaches \\(h\\) (via \\(a,b,e,d,h\\)); in fact, \\(a\\) reaches all the other vertices.\nThis does not hold for every vertex: \\(g\\) has no arc leaving it, so \\(g\\) reaches nobody, and in particular, \\(g\\) does not reach \\(a\\).\nConnectivity in Digraphs # The notion of connectivity, seen in Paths, Cycles, and Connectivity for ordinary graphs, is richer in digraphs, because directions matter:\nStrongly Connected Digraph \\(D\\) is strongly connected if for every pair \\(v, w \\in V\\) there is a directed path from \\(v\\) to \\(w\\) and from \\(w\\) to \\(v\\).\nUnilaterally Connected Digraph \\(D\\) is unilaterally connected if for every pair \\(v, w \\in V\\) there is a directed path from \\(v\\) to \\(w\\) or from \\(w\\) to \\(v\\) (at least one of the two).\nWeakly Connected Digraph \\(D\\) is weakly connected if the underlying graph is connected (ignoring the directions).\nDisconnected Digraph \\(D\\) is disconnected if the underlying graph is disconnected.\nConnectivity Hierarchy # $$\\text{Strongly connected} \\implies \\text{Unilaterally connected} \\implies \\text{Weakly connected}$$The implications are strict: each level is weaker, as the following examples show.\nExample: strongly connected Problem: Verify that the digraph \\(D\\) with \\(V=\\{a,b,c,d,e\\}\\) and arcs \\(E = \\{(a,b),(b,c),(c,d),(d,a),(a,e),(c,e),(e,b),(e,d)\\}\\), shown below, is strongly connected.\nEvery vertex reaches every other Solution: It suffices to show that, starting from any vertex, we reach all the others. For example, starting from \\(a\\): we reach \\(e\\) directly, \\(b\\) via \\(a,b\\), \\(c\\) via \\(a,b,c\\), and \\(d\\) via \\(a,e,d\\) (or \\(a,b,c,d\\)). Repeating this check from each vertex, we confirm that \\(D\\) is strongly connected.\nExample: the three levels of connectivity side by side Problem: Classify the three digraphs below, all with \\(V = \\{a,b,c,d,e\\}\\), by their level of connectivity:\n\\(D_1\\): arcs \\((a,b), (d,a), (b,c), (d,c), (c,e)\\). \\(D_2\\): arcs \\((a,b), (a,d), (c,b), (c,d), (c,e)\\). \\(D_3\\): arcs \\((a,b), (d,a), (b,c), (d,c)\\), with \\(e\\) isolated. D1, D2, and D3: three levels of connectivity Solution:\nIn \\(D_1\\), \\(d\\) is a source that reaches all the others (\\(d \\to a \\to b \\to c \\to e\\)), and \\(e\\) is a sink reached by all the others. Since \\(a\\), \\(b\\), and \\(c\\) form a direct chain \\(a \\to b \\to c\\), every pair of vertices has a path in at least one direction, and \\(D_1\\) is unilaterally connected. But since nothing arrives at \\(d\\) and \\(e\\) reaches nobody, \\(D_1\\) is not strongly connected. In \\(D_2\\), \\(a\\) only reaches \\(b\\) and \\(d\\), while \\(c\\) reaches \\(b\\), \\(d\\), and \\(e\\): neither reaches the other. This single pair is enough for \\(D_2\\) not to be unilaterally connected. But the underlying graph is connected (all vertices link up through \\(a\\) and \\(c\\)), so \\(D_2\\) is weakly connected. In \\(D_3\\), the vertex \\(e\\) is isolated: no arc links it to the others, not even ignoring direction. Since the underlying graph is already disconnected, \\(D_3\\) is disconnected. Checking with code Verifying connectivity by hand requires checking paths between many pairs of vertices, which is tiresome even in small digraphs. networkx has a ready-made function for each of the three levels: nx.is_strongly_connected, nx.is_semiconnected (that is the technical name for \u0026ldquo;unilaterally connected\u0026rdquo;), and nx.is_weakly_connected:\nimport networkx as nx D1 = nx.DiGraph([(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;a\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;)]) D2 = nx.DiGraph([(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;)]) D3 = nx.DiGraph([(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;a\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;c\u0026#34;)]) D3.add_node(\u0026#34;e\u0026#34;) # e isolated for name, D in [(\u0026#34;D1\u0026#34;, D1), (\u0026#34;D2\u0026#34;, D2), (\u0026#34;D3\u0026#34;, D3)]: print(f\u0026#34;{name}: strong={nx.is_strongly_connected(D)}, \u0026#34; f\u0026#34;unilateral={nx.is_semiconnected(D)}, \u0026#34; f\u0026#34;weak={nx.is_weakly_connected(D)}\u0026#34;) D1: strong=False, unilateral=True, weak=True D2: strong=False, unilateral=False, weak=True D3: strong=False, unilateral=False, weak=False Note that weak=True for \\(D_1\\) too: that makes sense, since unilaterally connected implies weakly connected (the hierarchy seen before). Only \\(D_3\\) has weak=False, precisely because \\(e\\) is isolated.\nStrongly connected components A digraph that is not strongly connected splits into strongly connected components: the largest sets of vertices in which each one reaches all the others. In the digraph of the walk examples, they are \\(\\{a,b,c,d,e,f\\}\\), \\(\\{g\\}\\), and \\(\\{h\\}\\). Finding them is a classic problem, solved in linear time, \\(O(n + m)\\), by two algorithms based on depth-first search: Robert Tarjan\u0026rsquo;s, published in 1972, and S. Rao Kosaraju\u0026rsquo;s, described in 1978 and published by Micha Sharir in 1981 (see the entry Strongly connected component). In networkx, the function is nx.strongly_connected_components:\nimport networkx as nx D = nx.DiGraph([ (\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;a\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;a\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;h\u0026#34;), ]) print(sorted(sorted(c) for c in nx.strongly_connected_components(D))) # Output: [[\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;e\u0026#39;, \u0026#39;f\u0026#39;], [\u0026#39;g\u0026#39;], [\u0026#39;h\u0026#39;]] Matrix Representation # Adjacency Matrix of a Digraph # Adjacency Matrix For \\(D\\) with \\(V = \\{v_1, \\ldots, v_n\\}\\), the adjacency matrix \\(A = (a_{ij})\\) is \\(n \\times n\\): $$a_{ij} = \\begin{cases} 1 \u0026 \\text{if } (v_i, v_j) \\in E(D) \\\\ 0 \u0026 \\text{otherwise} \\end{cases}$$ Warning The adjacency matrix of a digraph is not symmetric in general, unlike that of undirected graphs.\nRelation to degrees:\nSum of row \\(i\\): gives \\(d^+(v_i)\\) (out-degree) Sum of column \\(j\\): gives \\(d^-(v_j)\\) (in-degree) Example: adjacency matrix Problem: Build the adjacency matrix of the digraph \\(D\\) with \\(V = \\{v_1,v_2,v_3,v_4\\}\\) and arcs \\((v_1,v_2), (v_1,v_4), (v_3,v_2), (v_3,v_4), (v_4,v_2), (v_2,v_4)\\), and identify sources and sinks from it.\nDigraph D: V={v1,v2,v3,v4} Solution: $$A = \\begin{array}{c|cccc} \u0026 v_1 \u0026 v_2 \u0026 v_3 \u0026 v_4 \\\\ \\hline v_1 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \\\\ v_2 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \\\\ v_3 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \\\\ v_4 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\end{array}$$ Row of \\(v_1\\): sum = 2 = \\(d^+(v_1)\\) Column of \\(v_2\\): sum = \\(1+0+1+1 = 3 = d^-(v_2)\\) The columns of \\(v_1\\) and \\(v_3\\) are entirely zero: \\(v_1\\) and \\(v_3\\) are sources (no arc arrives at them) No row is entirely zero: \\(D\\) has no sink Incidence Matrix of a Digraph # Incidence Matrix For \\(D\\) with \\(n\\) vertices and \\(m\\) arcs, the incidence matrix \\(B = (b_{ij})\\) is \\(n \\times m\\): $$b_{ij} = \\begin{cases} +1 \u0026 \\text{if the arc } e_j \\text{ diverges from } v_i \\text{ (tail)} \\\\ -1 \u0026 \\text{if the arc } e_j \\text{ converges to } v_i \\text{ (head)} \\\\ 0 \u0026 \\text{otherwise} \\end{cases}$$Each column has exactly one \\(+1\\) (the tail of the arc) and one \\(-1\\) (the head of the arc).\nExample: incidence matrix Problem: Build the incidence matrix of the same digraph \\(v_1,v_2,v_3,v_4\\) from the previous example, now with the arcs labeled \\(e_1=(v_1,v_2), e_2=(v_3,v_2), e_3=(v_3,v_4), e_4=(v_1,v_4), e_5=(v_4,v_2), e_6=(v_2,v_4)\\):\nDigraph D with arcs labeled e1..e6 Solution: $$B = \\begin{array}{c|cccccc} \u0026 e_1 \u0026 e_2 \u0026 e_3 \u0026 e_4 \u0026 e_5 \u0026 e_6 \\\\ \\hline v_1 \u0026 1 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\\\ v_2 \u0026 -1 \u0026 -1 \u0026 0 \u0026 0 \u0026 -1 \u0026 1 \\\\ v_3 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \\\\ v_4 \u0026 0 \u0026 0 \u0026 -1 \u0026 -1 \u0026 1 \u0026 -1 \\end{array}$$Each column has exactly one \\(+1\\) and one \\(-1\\): for example, the column of \\(e_5\\) has \\(+1\\) at \\(v_4\\) and \\(-1\\) at \\(v_2\\), confirming the arc \\((v_4,v_2)\\).\nChecking with code networkx computes both matrices out of the box: nx.to_numpy_array for the adjacency matrix and nx.incidence_matrix for the incidence matrix. Just one detail: the sign convention of networkx is the opposite of ours (\\(-1\\) at the tail and \\(+1\\) at the head, instead of \\(+1\\) at the tail and \\(-1\\) at the head), so it is enough to multiply the result by \\(-1\\) to match the matrix \\(B\\) we computed by hand:\nimport networkx as nx D = nx.DiGraph() D.add_edges_from([(\u0026#34;v1\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v1\u0026#34;, \u0026#34;v4\u0026#34;), (\u0026#34;v3\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v3\u0026#34;, \u0026#34;v4\u0026#34;), (\u0026#34;v4\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v2\u0026#34;, \u0026#34;v4\u0026#34;)]) vertices = [\u0026#34;v1\u0026#34;, \u0026#34;v2\u0026#34;, \u0026#34;v3\u0026#34;, \u0026#34;v4\u0026#34;] arcs = [(\u0026#34;v1\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v3\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v3\u0026#34;, \u0026#34;v4\u0026#34;), (\u0026#34;v1\u0026#34;, \u0026#34;v4\u0026#34;), (\u0026#34;v4\u0026#34;, \u0026#34;v2\u0026#34;), (\u0026#34;v2\u0026#34;, \u0026#34;v4\u0026#34;)] # e1..e6 A = nx.to_numpy_array(D, nodelist=vertices, dtype=int) print(\u0026#34;Adjacency matrix:\\n\u0026#34;, A) B = -nx.incidence_matrix(D, nodelist=vertices, edgelist=arcs, oriented=True).toarray().astype(int) print(\u0026#34;Incidence matrix:\\n\u0026#34;, B) Adjacency matrix: [[0 1 0 1] [0 0 0 1] [0 1 0 1] [0 1 0 0]] Incidence matrix: [[ 1 0 0 1 0 0] [-1 -1 0 0 -1 1] [ 0 1 1 0 0 0] [ 0 0 -1 -1 1 -1]] Both match, arc by arc, the matrices \\(A\\) and \\(B\\) computed in the examples above.\nApplications of Digraphs # With the theoretical basis established, it is worth highlighting how digraphs show up in practice, often as a central part of other structures:\nNetwork flow: water in pipes with valves, electric current in circuits; the direction of the flow is part of the model The World Wide Web: pages are vertices and hyperlinks are arcs; PageRank (detailed in Graphs: A Language for Modeling Connections) uses the in- and out-degrees of this giant digraph to estimate the relevance of each page Call graphs: in software analysis, which functions call which; an arc \\((f,g)\\) means that \\(f\\) calls \\(g\\) DAGs (directed acyclic graphs): digraphs with no directed cycle, used to model dependencies DAG: Directed Acyclic Graph A DAG (Directed Acyclic Graph) is an acyclic digraph (with no directed cycles). It is used in:\nScheduling tasks with dependencies (the arc \\((A, B)\\) means \u0026ldquo;A before B\u0026rdquo;), as in Apache Airflow Installing packages with dependencies (in real managers, such as npm and pip, cycles are tolerated, and the DAG is the ideal case) Analyzing dependencies in Makefiles Representing genealogies Topological Sort # In a DAG of dependencies, the practical question is: in what order should we run the tasks, so that none runs before those it needs? The answer is a topological sort.\nTopological Sort A topological sort of a digraph \\(D\\) is a list of all its vertices in which, for every arc \\((u, v)\\), \\(u\\) appears before \\(v\\).\nTheorem A digraph admits a topological sort if and only if it is a DAG.\nProof: (\\(\\Rightarrow\\)) Suppose \\(D\\) has a topological sort and also a directed cycle. Among the vertices of the cycle, let \\(v\\) be the one that appears first in the sort. The arc of the cycle that arrives at \\(v\\) leaves from a vertex that appears after \\(v\\), which the sort forbids. Hence \\(D\\) has no cycles.\n(\\(\\Leftarrow\\)) First, every DAG has a source. If no vertex were a source, we could start from any vertex and always walk backward, along an arc that arrives at the current vertex; since there are finitely many vertices, one would repeat, and the stretch between the two visits would be a directed cycle. Now, by induction on the number of vertices: put a source \\(s\\) at the start of the list; the digraph \\(D - s\\) still has no cycles and, by the inductive hypothesis, has a topological sort, which comes after \\(s\\). No arc arrives at \\(s\\), so no arc is violated.\nTherefore, a digraph admits a topological sort if and only if it is a DAG, which is what we wanted to prove. \\(\\blacksquare\\)\nThe second half of the proof is already an algorithm, described by Arthur B. Kahn in 1962: remove a source, then another, and so on. To avoid searching for sources from scratch at each step, Kahn\u0026rsquo;s algorithm keeps the in-degree of each vertex and a queue with the current sources; when removing a vertex \\(u\\), it decreases by 1 the in-degree of each vertex that \\(u\\) points to, and any vertex that reaches 0 enters the queue. If the queue empties before all vertices have left, a cycle remains. Each vertex and each arc are handled once: \\(O(n + m)\\) time (see the entry Topological sorting).\nfrom collections import deque import networkx as nx tasks = nx.DiGraph([ (\u0026#34;extract\u0026#34;, \u0026#34;clean\u0026#34;), (\u0026#34;clean\u0026#34;, \u0026#34;train\u0026#34;), (\u0026#34;train\u0026#34;, \u0026#34;evaluate\u0026#34;), (\u0026#34;clean\u0026#34;, \u0026#34;plots\u0026#34;), (\u0026#34;evaluate\u0026#34;, \u0026#34;report\u0026#34;), (\u0026#34;plots\u0026#34;, \u0026#34;report\u0026#34;), ]) def topological_order(D): \u0026#34;\u0026#34;\u0026#34;Kahn\u0026#39;s algorithm: a topological sort of D, or None if D has a cycle.\u0026#34;\u0026#34;\u0026#34; in_degree = {v: D.in_degree(v) for v in D} queue = deque(v for v in D if in_degree[v] == 0) # the sources order = [] while queue: u = queue.popleft() order.append(u) for w in D.successors(u): in_degree[w] -= 1 if in_degree[w] == 0: queue.append(w) return order if len(order) == D.number_of_nodes() else None print(topological_order(tasks)) # Output: [\u0026#39;extract\u0026#39;, \u0026#39;clean\u0026#39;, \u0026#39;train\u0026#39;, \u0026#39;plots\u0026#39;, \u0026#39;evaluate\u0026#39;, \u0026#39;report\u0026#39;] print(list(nx.topological_sort(tasks))) # Output: [\u0026#39;extract\u0026#39;, \u0026#39;clean\u0026#39;, \u0026#39;train\u0026#39;, \u0026#39;plots\u0026#39;, \u0026#39;evaluate\u0026#39;, \u0026#39;report\u0026#39;] tasks.add_edge(\u0026#34;report\u0026#34;, \u0026#34;extract\u0026#34;) # closes a cycle print(topological_order(tasks)) # None A DAG usually has several topological sorts (here, plots could come before train), and any of them is a valid order of execution. The figure below shows the example DAG and, below it, the same vertices in a line, in the order found:\nIn a topological sort, all arcs point forward Application: Apache Airflow Apache Airflow is a tool for orchestrating data pipelines and models each pipeline literally as a DAG: each task is a vertex, and the \u0026gt;\u0026gt; operator declares a dependency arc between two tasks.\nfrom datetime import datetime from airflow import DAG from airflow.operators.python import PythonOperator def extract_data(): print(\u0026#34;extracting the sales data\u0026#34;) def transform_data(): print(\u0026#34;transforming the data\u0026#34;) def load_data(): print(\u0026#34;loading the data into the database\u0026#34;) with DAG(\u0026#34;sales_pipeline\u0026#34;, start_date=datetime(2026, 1, 1), schedule=\u0026#34;@daily\u0026#34;) as dag: extract = PythonOperator(task_id=\u0026#34;extract_data\u0026#34;, python_callable=extract_data) transform = PythonOperator(task_id=\u0026#34;transform_data\u0026#34;, python_callable=transform_data) load = PythonOperator(task_id=\u0026#34;load_data\u0026#34;, python_callable=load_data) extract \u0026gt;\u0026gt; transform \u0026gt;\u0026gt; load # DAG arcs: extract -\u0026gt; transform -\u0026gt; load Translating to the language of this article: each PythonOperator is a vertex (a task that runs a Python function); the with DAG(...) block declares the digraph itself, with a name and a run frequency; and the line extract \u0026gt;\u0026gt; transform \u0026gt;\u0026gt; load declares the arcs \\((\\text{extract}, \\text{transform})\\) and \\((\\text{transform}, \\text{load})\\), in the order in which the tasks need to run. At execution time, Airflow follows a topological sort of this DAG.\nIf someone tried to close the cycle with load \u0026gt;\u0026gt; extract, Airflow would reject the DAG at validation: acyclicity is a rule the tool enforces.\nComparison: Graph vs. Digraph # Feature Graph Digraph Edges/Arcs Unordered pairs \\(\\{u,v\\}\\) Ordered pairs \\((u,v)\\) Degree \\(d(v)\\) (single) \\(d^+(v)\\) (out) and \\(d^-(v)\\) (in) Special vertices n/a Source (\\(d^- = 0\\)), Sink (\\(d^+ = 0\\)) Adjacency matrix Symmetric Not necessarily symmetric Connectivity Connected / Disconnected Strong / Unilateral / Weak / Disconnected Handshaking \\(\\sum d(v) = 2|E|\\) \\(\\sum d^+(v) = \\sum d^-(v) = |E|\\) Summary Table # Concept Expression/Definition Digraph \\(D = (V,E)\\), with \\(E\\) made of ordered pairs (arcs) In-degree / out-degree \\(d^-(v) = |\\{(u,v) \\in E(D)\\}|\\), \\(d^+(v) = |\\{(v,u) \\in E(D)\\}|\\) Source / Sink \\(d^-(v) = 0\\) (source) / \\(d^+(v) = 0\\) (sink) Handshaking for digraphs \\(\\sum_{v} d^-(v) = \\sum_{v} d^+(v) = |E(D)|\\) Underlying graph \\(D\\) without directions, without parallel edges Orientation \\(\\vec{G}\\): digraph obtained by choosing a direction for each edge of \\(G\\) (not unique) Strongly connected Every pair \\(v,w\\) has a path from \\(v\\) to \\(w\\) and from \\(w\\) to \\(v\\) Unilaterally connected Every pair \\(v,w\\) has a path from \\(v\\) to \\(w\\) or from \\(w\\) to \\(v\\) Weakly connected The underlying graph is connected Disconnected The underlying graph is disconnected Adjacency matrix \\(a_{ij} = 1\\) if \\((v_i,v_j) \\in E(D)\\), \\(0\\) otherwise Incidence matrix \\(b_{ij} = +1\\) (tail), \\(-1\\) (head), \\(0\\) otherwise Strongly connected components Largest sets of mutually reachable vertices; Tarjan and Kosaraju in \\(O(n+m)\\) Topological sort Exists \\(\\iff\\) \\(D\\) is a DAG; Kahn\u0026rsquo;s algorithm in \\(O(n+m)\\) Exercises # Exercise 1: Sources and sinks For the digraph \\(D\\) with vertices \\(\\{a,b,c,d\\}\\) and arcs \\(\\{(a,b),(a,c),(c,a),(c,b),(d,a),(d,c)\\}\\):\n(a) Which are the sources? (b) Which are the sinks? (c) Does \\(a\\) reach all vertices? (d) Is \\(D\\) strongly connected?\nSolution (a) Source: \\(d\\), since no arc arrives at \\(d\\) (\\(d^-(d) = 0\\)).\n(b) Sink: \\(b\\), since no arc leaves \\(b\\) (\\(d^+(b) = 0\\)).\n(c) From \\(a\\): \\(a \\to b\\) ✓, \\(a \\to c\\) ✓, but \\(a\\) does not reach \\(d\\). False.\n(d) No, since \\(d\\) is a source and no vertex reaches \\(d\\). Hence \\(D\\) is not strongly connected.\nExercise 2: Connectivity Classify as strongly, unilaterally, or weakly connected:\n(a) Linear chain \\(a \\to b \\to c \\to d \\to e \\to f\\) (b) Cycle \\(a \\to b \\to c \\to a\\) with extra arc \\(b \\to d\\)\nSolution (a) Unilaterally connected: from \\(a\\) we reach all vertices. For any pair \\((v_i, v_j)\\) with \\(i \u0026lt; j\\), there is a path from \\(v_i\\) to \\(v_j\\). But \\(f\\) does not reach \\(a\\): it is not strongly connected.\n(b) The vertex \\(d\\) has no arc leaving it, it is a sink and reaches nobody. Since \\(d\\) does not reach any other vertex, the digraph is not strongly connected. But every vertex reaches \\(d\\) (\\(a \\to b \\to d\\), \\(b \\to d\\), \\(c \\to a \\to b \\to d\\)), and among \\(a\\), \\(b\\), and \\(c\\) the cycle guarantees a path in both directions, so every pair has a path in at least one direction, and the digraph is unilaterally connected.\nFinal Series Recap # With digraphs, we have completed the fundamentals of graph theory. Recapping \u0026ldquo;The Mathematics of Connections\u0026rdquo;:\nArticle Central Topic Introduction History, Königsberg, applications Definitions \\(G=(V,E)\\), subgraphs, \\(K_n\\), cliques Degree \\(d(v)\\), regular graphs, Handshaking Isomorphism Structure-preserving bijections, matrices Paths Walk/trail/path/cycle, connectivity, BFS and Dijkstra, bipartite Trees Acyclic+connected, \\(m=n-1\\), center, spanning Eulerian/Hamiltonian Special tours, Traveling Salesman Problem (TSP) Planar Euler\u0026rsquo;s formula, \\(K_5\\) and \\(K_{3,3}\\), Kuratowski, coloring and Four Colors Digraphs Broken symmetry, oriented degrees, DAGs, topological sort This is the last article of the series \u0026ldquo;The Mathematics of Connections\u0026rdquo;.\n","date":"2026-08-27","externalUrl":null,"permalink":"/2026/08/directed-graphs/","section":"Blog","summary":"A one-way street, a hyperlink, and a dependency between tasks are relations with a direction, and directed graphs exist for them. We will see in- and out-degrees, strong, unilateral, and weak connectivity, matrices, DAGs, and the topological sort that Apache Airflow uses.","title":"Directed Graphs: When Direction Matters","type":"posts"},{"content":"","date":"2026-08-27","externalUrl":null,"permalink":"/categories/discrete-mathematics/","section":"Categories","summary":"","title":"Discrete Mathematics","type":"categories"},{"content":"","date":"2026-08-27","externalUrl":null,"permalink":"/tags/graph-theory/","section":"tags","summary":"","title":"graph theory","type":"tags"},{"content":"","date":"2026-08-27","externalUrl":null,"permalink":"/categories/matematica-discreta/","section":"Categorias","summary":"","title":"Matemática Discreta","type":"categories"},{"content":"","date":"2026-08-27","externalUrl":null,"permalink":"/tags/teoria-dos-grafos/","section":"tags","summary":"","title":"teoria dos grafos","type":"tags"},{"content":"A complete introduction to Graph Theory: from the Bridges of Königsberg to directed graphs. Learn the mathematics behind the networks, algorithms and connections of the digital world: degree, isomorphism, paths, trees, planarity and more.\nThe articles are listed below, in the order they were written.\n","date":"2026-08-27","externalUrl":null,"permalink":"/series/the-mathematics-of-connections/","section":"Series","summary":"","title":"The Mathematics of Connections","type":"series"},{"content":" Have you ever tried to connect three houses to three utilities (gas, water, and electricity) using lines that do not cross? If so, you found a mathematical result in practice: this task is impossible.\nThe branch of graph theory that studies when connections can be made without crossings is called planarity theory. It has practical consequences in printed circuit board design and in network visualization.\nWhy Do Planar Graphs Matter? # Knowing whether a graph can be drawn without crossings is a question with practical weight in computer science and engineering:\nPrinted circuit board (PCB) and VLSI design: a copper trace that crosses another on the same layer causes a short circuit. Determining whether a circuit is planar decides whether it fits on a single layer or needs extra vias and layers. Network visualization: software dependency diagrams, social network maps, and flowcharts are much more readable when drawn without crossings, and graph drawing algorithms use planarity tests as a first step. Maps and the Four Color Theorem: every political map can be represented by a planar graph (each region becomes a vertex, each shared border becomes an edge), and it is this planarity that turns the Four Color Theorem into a problem about graphs, treated in the coloring section of this article. Transportation and utility networks: the three houses and three utilities problem, below, is the same as deciding whether a network of pipes, cables, or rails can be installed without crossings. The Three Houses and Three Utilities Problem # The classic puzzle asks us to connect 3 houses (\\(h_1, h_2, h_3\\)) to 3 utilities (gas \\(g\\), water \\(w\\), electricity \\(e\\)), for safety reasons, say, with none of the 9 connection lines crossing. Try to draw 8 of the connections without crossings: it is easy. Now try to add the last one, between the leftover house and the leftover utility: no matter how you rearrange the drawing, that last line always crosses one of the other eight.\nWe can model this problem as a graph, with one vertex for each house and each utility, and one edge for each of the 9 connections. The following figure shows two representations of this graph: \\(G_1\\), with the 3 houses on one side and the 3 utilities on the other (the natural \u0026ldquo;bipartite\u0026rdquo; drawing of the statement), and \\(G_2\\), with the same 6 vertices placed on a hexagon, alternating houses and utilities. The two drawings look different, but \\(G_1\\) and \\(G_2\\) are isomorphic: they are the same graph, drawn in different ways.\nG1 (bipartite layout) and G2 (hexagonal layout): two representations of the same graph K₃,₃, the graph of the three houses and three utilities problem. This graph is a complete bipartite graph: each house is connected to all 3 utilities, and vice versa. It is denoted \\(K_{3,3}\\).\nMathematically, the three houses and three utilities problem is then equivalent to asking: can we draw the graph \\(K_{3,3}\\) so that no edge crosses another? In other words, is the complete bipartite graph \\(K_{3,3}\\) planar? The answer, which we will prove later, is no.\nPlanar Graphs # Planar Graph A graph \\(G\\) is planar if it can be drawn in the plane so that no pair of edges crosses, except at vertices. Such a drawing is called a plane representation (or plane embedding) of \\(G\\) (see the entry Planar graph).\nExample 1: the complete graph \\(K_4\\) Statement: The most obvious drawing of \\(K_4\\) (a square with both diagonals) has crossing edges. Does this mean that \\(K_4\\) is non-planar?\nK4: one non-planar representation (crossing diagonals) and three equivalent planar representations, all without crossing edges. Solution: No. The figure shows this non-planar representation next to three equivalent planar representations of the same graph \\(K_4\\), all without crossing edges, just by rearranging the vertices. Hence \\(K_4\\) is planar: at least one plane representation of it exists, even if other drawings of the same graph have crossings.\nExample 2: the graphs \\(K_5\\) and \\(K_{3,3}\\) Statement: Is it possible to draw a plane representation of the graphs \\(K_5\\) (the complete graph on 5 vertices) and \\(K_{3,3}\\) (the graph of the utilities problem, from the previous section)?\nSolution: No. However you reposition the vertices, at least one crossing always remains; the following figure shows both graphs drawn this way:\nK5 (pentagon with all diagonals) and K3,3 (bipartite layout), both drawn with crossing edges; no plane representation exists for either one. We will rigorously prove later, using Euler\u0026rsquo;s Formula, that neither of them admits a plane representation.\nOther immediate examples:\nTrees: always planar, since they can be drawn without crossings starting from a root, level by level, with the children of each vertex side by side right below it \\(C_n\\) (cycles): planar for every \\(n\\) Faces of a Planar Graph # When we draw a planar graph in the plane, the edges divide the plane into regions called faces.\nFaces If \\(G\\) is a planar graph, every plane representation of \\(G\\) divides the plane into regions called faces. One of these regions is unbounded and is called the outer face.\nAn important property: if \\(G\\) is a planar graph, every plane representation of \\(G\\) has the same number of faces, denoted \\(f\\). In other words, although the drawing of a planar graph is not unique, the number of faces is an invariant of the graph: no matter how you draw it, the number of regions created is always the same.\nExample: three representations of \\(K_4\\), always with 4 faces Statement: Consider \\(K_4\\) (\\(n = 4\\) vertices, \\(m = 6\\) edges) and three different plane representations: \\(R_1\\), an outer triangle with an inner vertex connected to the three corners; \\(R_2\\), a square with a diagonal and an outer arc; \\(R_3\\), four vertices in a line with arc edges above and below. Do all three have the same number of faces?\nR1, R2, and R3: three different plane representations of K4, each with its faces numbered. In all of them, f = 4. Solution: Yes. Although they are completely different drawings of the same graph, the three representations have exactly \\(f = 4\\) faces; in \\(R_1\\), for example, there are 3 inner (triangular) faces plus 1 outer face. This coincidence is no accident: it is exactly what Euler\u0026rsquo;s Formula, below, formalizes.\nEuler\u0026rsquo;s Formula # Theorem: Euler\u0026rsquo;s Formula If \\(G\\) is a connected planar graph with \\(n\\) vertices, \\(m\\) edges, and \\(f\\) faces, then: $$n - m + f = 2$$Proof:\nConsider a spanning tree \\(T\\) of \\(G\\) (removing \\(m - (n-1)\\) edges):\n\\(T\\) has \\(n\\) vertices, \\(n-1\\) edges, 1 face (the whole plane): \\(n - (n-1) + 1 = 2\\). ✓ Now, we add an edge back to the graph. Each added edge connects two vertices that are already in the drawing and creates exactly one new face (it splits an existing face in two):\n\\(\\Delta n = 0\\), \\(\\Delta m = +1\\), \\(\\Delta f = +1\\) The equation \\(n - m + f = 2\\) still holds Adding all the remaining \\(m - (n-1)\\) edges, we arrive at \\(G\\) with the equality preserved. Therefore, \\(n - m + f = 2\\) for every connected planar graph, which is what we wanted to prove. \\(\\blacksquare\\)\nThe formula is the planar case of a more general invariant of topology, the Euler characteristic.\nExample: building the graph from the spanning tree Statement: What does the construction used in the proof above look like, step by step: start from a spanning tree \\(T_G\\) of a graph \\(G\\) and put the missing edges back, one by one?\nSolution: The following figure illustrates the sequence: \\(T_G\\) has only 1 face, the whole plane. Each edge put back connects two vertices that already exist in the drawing and cuts an existing face in two: that is why \\(n\\) stays fixed while \\(m\\) and \\(f\\) increase together, always by 1, preserving \\(n - m + f = 2\\) at every step.\nG, its spanning tree T_G (f=1), and the reconstruction of G by adding the 3 remaining edges one by one (highlighted in red), with n-m+f=2 preserved at every step. Check with \\(K_4\\) \\(n = 4\\), \\(m = 6\\): \\(4 - 6 + f = 2 \\implies f = 4\\). Confirmed! ✓\nWhat if the graph is disconnected? If \\(G\\) is planar with \\(c\\) connected components (the \\(\\omega(G)\\) from the article on paths), the formula becomes \\(n - m + f = 1 + c\\). Applying Euler\u0026rsquo;s Formula to each component and adding up, we get \\(n - m + (\\text{sum of the faces}) = 2c\\); but the outer face was counted \\(c\\) times, once per component, and it is only one. Removing the \\(c - 1\\) repetitions, \\(n - m + f = 2c - (c - 1) = 1 + c\\). With \\(c = 1\\), we are back to \\(n - m + f = 2\\).\nCorollaries: Bounds for Planar Graphs # Euler\u0026rsquo;s Formula implies important inequalities that bound the number of edges in planar graphs.\nCorollary 1: General Bound # Corollary 1 If \\(G\\) is connected, planar, and \\(n \\geq 3\\), then: $$m \\leq 3n - 6$$Proof: Each face is bounded by at least 3 edges. Each edge bounds at most 2 faces. Hence: $$3f \\leq 2m \\implies f \\leq \\frac{2m}{3}$$ From Euler\u0026rsquo;s Formula: \\(f = 2 - n + m\\). Substituting: $$3(2 - n + m) \\leq 2m \\implies 6 - 3n + 3m \\leq 2m \\implies m \\leq 3n - 6$$Therefore, every connected planar graph with \\(n \\geq 3\\) vertices has \\(m \\leq 3n - 6\\), which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary 2: Triangle-Free Graphs # Corollary 2 If \\(G\\) is connected, planar, \\(n \\geq 3\\), and triangle-free (no cycles of length 3), then: $$m \\leq 2n - 4$$Proof: Without triangles, each face is bounded by at least 4 edges. Hence: $$4f \\leq 2m \\implies 4(2-n+m) \\leq 2m \\implies m \\leq 2n - 4$$Therefore, every connected planar triangle-free graph with \\(n \\geq 3\\) vertices has \\(m \\leq 2n - 4\\), which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary 3: \\(K_5\\) Is Not Planar # \\(K_5\\) has \\(n = 5\\), \\(m = 10\\). If it were planar: \\(m \\leq 3(5) - 6 = 9\\). But \\(10 \u0026gt; 9\\). Contradiction. Therefore, \\(K_5\\) is not planar, which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary 4: \\(K_{3,3}\\) Is Not Planar # \\(K_{3,3}\\) has \\(n = 6\\) vertices, \\(m = 9\\) edges, and is bipartite: it contains no triangles.\nSuppose \\(K_{3,3}\\) is planar and let us try applying Corollary 1 first: the relation requires \\(m \\leq 3n - 6\\). But \\(m = 9 \u0026lt; 3(6) - 6 = 12\\), so the inequality is satisfied and Corollary 1 alone does not let us conclude anything about the planarity of \\(K_{3,3}\\).\nSince \\(K_{3,3}\\) has no triangles, we can apply Corollary 2, which is more restrictive: the relation requires \\(m \\leq 2n - 4\\). But \\(m = 9 \u0026gt; 2(6) - 4 = 8\\). Contradiction.\nTherefore, \\(K_{3,3}\\) is not planar, which is what we wanted to prove. \\(\\blacksquare\\)\nThe corollaries are necessary conditions, not sufficient ones The inequalities \\(m \\leq 3n - 6\\) and \\(m \\leq 2n - 4\\) are necessary for planarity, but not sufficient. There are graphs that satisfy these restrictions and still are not planar: that is precisely what happened above with \\(K_{3,3}\\) and Corollary 1.\nGraphs That Contain \\(K_5\\) or \\(K_{3,3}\\) as a Subgraph # Two simple but useful observations follow from everything we have seen so far.\nObservation 1: \\(K_5\\) and \\(K_{3,3}\\) are, respectively, the non-planar graphs with the fewest possible vertices and the fewest possible edges: proven minima, not just the smallest known. As we will see with Kuratowski\u0026rsquo;s Theorem, every non-planar graph contains a subdivision of one of the two and, therefore, has at least 5 vertices and at least 9 edges.\nObservation 2: If \\(G\\) is a planar graph, then every subgraph of \\(G\\) is also planar: removing vertices or edges from a drawing without crossings cannot create a crossing. Equivalently:\nObservation 2' If a graph \\(G\\) contains a non-planar graph as a subgraph, then \\(G\\) is non-planar.\nThis observation is already enough to detect non-planarity in graphs much larger than \\(K_5\\) or \\(K_{3,3}\\): we just need to find one of these two structures \u0026ldquo;hidden\u0026rdquo; inside a larger graph.\nExample: two non-planar graphs because they contain \\(K_5\\) and \\(K_{3,3}\\) Statement: Consider two graphs \\(G_1\\) and \\(G_2\\):\n\\(G_1\\) is formed by a copy of \\(K_5\\) (highlighted) with an extra vertex and an extra edge hanging from one of the vertices. \\(G_2\\) is formed by a copy of \\(K_{3,3}\\) (highlighted) with two extra vertices hanging. Are \\(G_1\\) and \\(G_2\\) planar?\nG1 (K5 highlighted + 1 extra vertex/edge) and G2 (K3,3 highlighted + 2 extra vertices): in both, the forbidden graph appears directly as a subgraph, with no subdivision. Solution: No, neither of them. Since \\(G_1\\) contains \\(K_5\\) as a subgraph, \\(G_1\\) is not planar. Since \\(G_2\\) contains \\(K_{3,3}\\) as a subgraph, \\(G_2\\) is not planar. Note that, in this example, the forbidden graph appears directly as a subgraph, without any subdivision. The next section generalizes this idea to the case where the forbidden graph appears \u0026ldquo;disguised\u0026rdquo; by subdivisions.\nGraph Subdivision # Subdivision A subdivision of \\(G\\) is obtained by inserting degree-2 vertices into the edges of \\(G\\), so each edge may be replaced by a path of arbitrary length.\nIntuitively, subdividing an edge means placing one or more intermediate \u0026ldquo;waypoints\u0026rdquo; on it, without changing the connection structure of the graph. The following figure shows a graph \\(G\\) and a subdivision \\(G^{\\prime}\\) of it, with degree-2 vertices (in red) inserted in varying numbers on each edge.\nG (square abcd with a diagonal) and G\u0026rsquo; (the same structure, with degree-2 vertices inserted on each edge, in varying numbers). Observation 3: If \\(G\\) is planar, then any subdivision of \\(G\\) is also planar: inserting a vertex in the middle of an edge cannot create a crossing. Equivalently:\nObservation 3' If \\(G\\) is a subdivision of a non-planar graph, then \\(G\\) is non-planar.\nThe following figure illustrates this observation with subdivisions of \\(K_5\\) and \\(K_{3,3}\\), with the inserted degree-2 vertices highlighted in red:\nSubdivisions of K5 and K3,3: degree-2 vertices (red) inserted into some edges. Even disguised, these structures remain non-planar. Combining Observation 2\u0026rsquo; (subgraph, from the previous section) with Observation 3\u0026rsquo; (subdivision), we obtain the more general result:\nObservation 4: If \\(G\\) is a graph that contains a subdivision of \\(K_5\\) or of \\(K_{3,3}\\) as a subgraph, then \\(G\\) is non-planar.\nKuratowski\u0026rsquo;s Theorem # The deepest result about planar graphs provides a complete characterization:\nKuratowski\u0026rsquo;s Theorem (1930) A graph \\(G\\) is planar if and only if it contains no subdivision of \\(K_5\\) or of \\(K_{3,3}\\) as a subgraph (see the entry Kuratowski\u0026rsquo;s theorem).\nThe direction \u0026ldquo;if \\(G\\) contains a subdivision of \\(K_5\\) or \\(K_{3,3}\\), then \\(G\\) is not planar\u0026rdquo; is already proven: it is exactly Observation 4 from the previous section. The hard and surprising part of the theorem is the converse: every graph that contains no such subdivisions is necessarily planar. That direction requires a long and technical proof, which is beyond the scope of this article.\nThis theorem says that \\(K_5\\) and \\(K_{3,3}\\) are, in a sense, the only obstacles to planarity. Every non-planar graph contains one of these two graphs \u0026ldquo;hidden\u0026rdquo; (possibly with subdivisions).\nAlthough the theorem provides the complete theoretical characterization, checking computationally whether a graph is planar led to the development of sophisticated algorithms. The first linear-time algorithm, \\(O(n)\\), was published by John Hopcroft and Robert Tarjan in 1974; in 2004, John Boyer and Wendy Myrvold arrived at a simpler linear algorithm, which also builds the plane drawing (see the entry Planarity testing). In networkx, a linear test is ready-made as nx.check_planarity.\nColoring Planar Graphs # In the first article of the series, the Four Color Theorem appeared as a statement about maps. With the language of this article, it becomes a theorem about planar graphs, and Euler\u0026rsquo;s Formula is enough to prove a weaker version of it.\nColoring and Chromatic Number A coloring of a graph \\(G\\) with \\(k\\) colors assigns to each vertex one of \\(k\\) colors, so that adjacent vertices get different colors. The chromatic number \\(\\chi(G)\\) is the smallest \\(k\\) for which a coloring of \\(G\\) with \\(k\\) colors exists.\nSome immediate consequences of the definition:\nVertices of the same color cannot be neighbors: they form an independent set. Coloring \\(G\\) with \\(k\\) colors is partitioning \\(V(G)\\) into \\(k\\) independent sets. If \\(G\\) contains a clique with \\(k\\) vertices, then \\(\\chi(G) \\geq k\\), because the vertices of the clique need all-different colors. In particular, \\(\\chi(K_n) = n\\). A graph with at least one edge has \\(\\chi(G) = 2\\) exactly when it is bipartite: the two colors are the two sides of the bipartition. That is why an odd cycle needs 3 colors. From maps to planar graphs. Each region of a map becomes a vertex, and two regions with a common border of positive length become adjacent vertices. The resulting graph is planar: just put each vertex inside its region and connect two neighboring regions by a line that crosses the common border. Coloring the map is coloring this graph, and the theorem takes its graph theory form:\nFour Color Theorem Every planar graph \\(G\\) has \\(\\chi(G) \\leq 4\\).\nA proof more than a century in the making In 1879, Alfred Kempe published a proof that was accepted for more than ten years, until Percy Heawood found an error in it, in 1890, and, in the same paper, proved that five colors always suffice. The proof by Kenneth Appel and Wolfgang Haken, from 1976, reduced the problem to a huge set of configurations checked by computer. In 1996, Neil Robertson, Daniel Sanders, Paul Seymour, and Robin Thomas presented a simpler proof, still with a computer, with 633 configurations; and in 2005 Georges Gonthier and Benjamin Werner formalized a complete proof in the Coq proof assistant (see the entry Four color theorem).\nFour colors without four mutually neighboring regions. The wheel with 5 spokes (a cycle \\(C_5\\) plus a central vertex connected to all five) is planar and does not contain \\(K_4\\): a copy of \\(K_4\\) would need three mutually adjacent vertices of the cycle, and \\(C_5\\) has no triangles. Even so, \\(\\chi = 4\\). The cycle \\(C_5\\) is odd and needs 3 colors, and the central vertex, a neighbor of all the vertices of the cycle, cannot repeat any of them. It is the graph version of the example that the first article used to show that \u0026ldquo;not having \\(K_5\\)\u0026rdquo; does not, by itself, explain why four colors suffice: needing \\(k\\) colors does not require containing \\(K_k\\). The figure below shows the map and the wheel side by side, with a 4-coloring:\nA region surrounded by a ring of five: no K₄, but χ = 4 The proof of the Four Color Theorem is beyond the reach of an article, but the six-color version follows from Corollary 1 in a few lines. The starting point is a lemma.\nLemma Every planar graph has a vertex of degree at most 5.\nProof: Let \\(H\\) be a connected component of the planar graph \\(G\\); \\(H\\) is also planar. If \\(H\\) has at most 2 vertices, its vertices have degree at most 1. Otherwise, \\(H\\) is connected, planar, and has \\(n_H \\geq 3\\) vertices, and Corollary 1 gives \\(m_H \\leq 3n_H - 6\\). If every vertex of \\(H\\) had degree at least 6, the Handshaking Theorem would give \\(2m_H = \\sum d(v) \\geq 6n_H\\), that is, \\(m_H \\geq 3n_H \u0026gt; 3n_H - 6\\), a contradiction. Therefore, \\(H\\), and with it \\(G\\), has a vertex of degree at most 5, which is what we wanted to prove. \\(\\blacksquare\\)\nSix Color Theorem Every planar graph \\(G\\) has \\(\\chi(G) \\leq 6\\).\nProof by induction on the number \\(n\\) of vertices:\nBase case: if \\(n \\leq 6\\), just give a different color to each vertex.\nInductive hypothesis: suppose every planar graph with \\(n - 1\\) vertices (\\(n \\geq 7\\)) can be colored with 6 colors.\nInductive step: let \\(G\\) be planar with \\(n\\) vertices. By the lemma, \\(G\\) has a vertex \\(v\\) with \\(d(v) \\leq 5\\). Removing \\(v\\) and its edges, we get \\(G - v\\), which is planar (it is a subgraph of \\(G\\), Observation 2) and has \\(n - 1\\) vertices; by the inductive hypothesis, \\(G - v\\) can be colored with 6 colors. The at most 5 neighbors of \\(v\\) use at most 5 of these colors, and at least one is left for \\(v\\).\nTherefore, every planar graph with \\(n\\) vertices can be colored with 6 colors, that is, \\(\\chi(G) \\leq 6\\), which is what we wanted to prove. \\(\\blacksquare\\)\nWith more work (swapping colors along paths of two alternating colors, the \u0026ldquo;Kempe chains\u0026rdquo;), the same lemma gives Heawood\u0026rsquo;s Five Color Theorem. To get to four, nobody has found an argument of that size.\nChecking with code networkx tests planarity with nx.check_planarity and colors with nx.greedy_color, which goes through the vertices in some order and gives each one the smallest color not yet used by its neighbors. Greedy coloring does not guarantee the minimum, but with the \u0026quot;smallest_last\u0026quot; strategy it repeats the Six Color proof: it removes a vertex of minimum degree, then another, and colors in the reverse order. That is why, on a planar graph, this strategy never uses more than 6 colors.\nimport networkx as nx wheel = nx.wheel_graph(6) # vertex 0 in the center, connected to the cycle 1-2-3-4-5 print(nx.check_planarity(wheel)[0]) # True colors = nx.greedy_color(wheel, strategy=\u0026#34;largest_first\u0026#34;) print(colors) # {0: 0, 1: 1, 2: 2, 3: 1, 4: 2, 5: 3} print(len(set(colors.values()))) # 4 icosahedron = nx.icosahedral_graph() # planar, all vertices with degree 5 print(nx.check_planarity(icosahedron)[0]) # True colors = nx.greedy_color(icosahedron, strategy=\u0026#34;smallest_last\u0026#34;) print(len(set(colors.values()))) # 4 The wheel uses exactly the four colors it needs. The icosahedron shows that the lemma cannot be improved (every vertex there has degree 5), and even so greedy coloring finds 4 colors.\nApplication: conflict-free schedules Five committees in a company need to meet, and two committees with a member in common cannot meet at the same time. In the conflict graph, each committee is a vertex and each shared member becomes an edge; the time slots are the colors, and the minimum number of time slots is \\(\\chi(G)\\). The same modeling distributes radio frequencies among neighboring antennas and processor registers among the variables of a program.\nimport networkx as nx conflicts = nx.Graph([ (\u0026#34;Budget\u0026#34;, \u0026#34;Personnel\u0026#34;), (\u0026#34;Budget\u0026#34;, \u0026#34;Construction\u0026#34;), (\u0026#34;Personnel\u0026#34;, \u0026#34;Construction\u0026#34;), (\u0026#34;Personnel\u0026#34;, \u0026#34;Technology\u0026#34;), (\u0026#34;Construction\u0026#34;, \u0026#34;Technology\u0026#34;), (\u0026#34;Technology\u0026#34;, \u0026#34;Events\u0026#34;), ]) schedules = nx.greedy_color(conflicts, strategy=\u0026#34;largest_first\u0026#34;) print(schedules) # Output: {\u0026#39;Personnel\u0026#39;: 0, \u0026#39;Construction\u0026#39;: 1, \u0026#39;Technology\u0026#39;: 2, \u0026#39;Budget\u0026#39;: 2, \u0026#39;Events\u0026#39;: 0} print(max(schedules.values()) + 1) # 3 time slots Three time slots are the minimum: Budget, Personnel, and Construction form a clique with three vertices, and a clique with \\(k\\) vertices requires \\(k\\) colors.\nSummary Table # Concept Expression/Definition Planar graph Drawable in the plane without crossing edges Face invariance Every plane representation of a connected graph has the same number of faces \\(f\\) Euler\u0026rsquo;s Formula \\(n - m + f = 2\\) (connected planar graph) General bound \\(m \\leq 3n - 6\\) (necessary; \\(n \\geq 3\\)) Triangle-free \\(m \\leq 2n - 4\\) (necessary; \\(n \\geq 3\\)) \\(K_5\\) \\(m = 10 \u0026gt; 9 = 3(5)-6\\) → non-planar \\(K_{3,3}\\) \\(m = 9 \u0026gt; 8 = 2(6)-4\\) → non-planar Planarity inheritance A subgraph of a planar graph is planar → a graph with a non-planar subgraph is non-planar Kuratowski Planar \\(\\iff\\) no subdivision of \\(K_5\\) or \\(K_{3,3}\\) as a subgraph General Euler\u0026rsquo;s Formula \\(n - m + f = 1 + c\\) (planar graph with \\(c\\) components) Chromatic number \\(\\chi(G)\\) = smallest number of colors with neighbors of different colors Minimum degree in planar graphs Every planar graph has a vertex of degree \\(\\leq 5\\) Six Color Theorem Planar graph \\(\\Rightarrow \\chi(G) \\leq 6\\) (proof by induction) Four Color Theorem Planar graph \\(\\Rightarrow \\chi(G) \\leq 4\\) (Appel and Haken, 1976) Exercises # Exercise 1: Number of faces A connected planar graph has 8 vertices and 12 edges. How many faces does it have?\nSolution From Euler\u0026rsquo;s Formula: \\(f = 2 - n + m = 2 - 8 + 12 = 6\\) faces.\nExercise 2: Is \\(K_{2,4}\\) planar? Statement: Is the complete bipartite graph \\(K_{2,4}\\) planar?\nSolution \\(K_{2,4}\\) has \\(n = 6\\), \\(m = 8\\). It is bipartite (triangle-free). By Corollary 2: \\(m \\leq 2(6) - 4 = 8\\). Since \\(8 \\leq 8\\), the corollary does not rule out planarity. In fact, \\(K_{2,4}\\) is planar: it is possible to draw it without crossings.\nExercise 3: non-planar subgraph A graph \\(G\\) has 8 vertices and 14 edges, and you discover that 5 of those vertices, together with 10 of the edges among them, form a copy of \\(K_5\\). What can be concluded about the planarity of \\(G\\)?\nSolution By Observation 2\u0026rsquo;, if \\(G\\) contains a non-planar graph (\\(K_5\\)) as a subgraph, then \\(G\\) is non-planar, regardless of what happens in the other 3 vertices and the other 4 edges.\nNext Steps # Everything in this article dealt with drawability: whether a graph can be placed in the plane without crossing edges. At no point did we ask whether the connections have a single direction: whether going from \\(A\\) to \\(B\\) is the same as going from \\(B\\) to \\(A\\). That is exactly the limitation that closes this stage of the series: planarity says nothing about the direction of the connections. In the next article of this series, we study directed graphs: when the direction of the connections matters.\n","date":"2026-08-25","externalUrl":null,"permalink":"/2026/08/planar-graphs/","section":"Blog","summary":"Try to connect three houses to gas, water, and electricity without crossing lines: it is impossible. We will see why, with Euler’s Formula, the proof that the three-houses graph and the complete graph on five vertices are not planar, Kuratowski’s Theorem, and the coloring of planar graphs, from six colors down to four.","title":"Planar Graphs: When Crossing Is Unavoidable","type":"posts"},{"content":" In Graphs: A Language for Modeling Connections, we saw that the Seven Bridges of Königsberg problem boils down to a parity argument: the four regions of the city (Altstadt, Kneiphof, Lomse, and Vorstadt) have degrees 3, 5, 3, and 3, respectively. Every region that is neither the start nor the end of a walk must have an even number of bridges; since all four regions of Königsberg have odd degree, the complete walk is impossible.\nThis informal argument hides two concepts that deserve their own names and a rigorous proof. First: what exactly does \u0026ldquo;traversing each edge exactly once\u0026rdquo; mean? A graph that admits this kind of traversal is called an Eulerian graph. Second: what if, instead of crossing every bridge, we only wanted to visit each region exactly once? This question, similar on the surface, is actually a different problem, and a much harder one, which gives rise to Hamiltonian graphs.\nIn this article, we formalize both concepts, prove Euler\u0026rsquo;s Theorem, which characterizes Eulerian graphs, and explore why the analogous question about vertices resists a simple characterization to this day, culminating in the Traveling Salesman Problem.\nWhy do Eulerian and Hamiltonian graphs matter? # Knowing whether a graph admits these two kinds of traversal is a recurring practical problem in computer science and engineering:\nUrban routing: finding a route that goes through each street of a neighborhood exactly once (garbage collection, mail delivery, snow removal, pipe inspection) is an Eulerian problem. When the graph is not Eulerian and some streets must be repeated, the problem becomes the Chinese Postman Problem, which looks for the shortest total route. Bioinformatics: assembling a genome from short DNA fragments is modeled with a De Bruijn graph, in which the assembly corresponds to an Eulerian trail; this idea is behind many short-read assemblers. Logistics and deliveries: deciding the order in which to visit a set of addresses while minimizing the distance traveled is the Traveling Salesman Problem, a Hamiltonian question. Circuit design: planning the path of a drill or solder probe that must visit each point of a printed circuit board exactly once. Task scheduling: sequencing operations on a machine that must go through each configuration of the system exactly once. The good news is that the Eulerian problem has a simple and efficient solution, as we will see next. The bad news is that the Hamiltonian problem (similar in formulation) is among the hardest that computer science knows.\nTwo problems, one graph # To appreciate the difference between the two problems, consider seven cities \\(a, b, c, d, e, f, g\\) connected by roads, represented by the graph below.\nSeven cities a, b, c, d, e, f, g connected by 12 roads. Vertex g sits in the center, connected to b, c, e, and f. The explorer problem. An explorer wants to travel each road exactly once, starting at \\(a\\) and returning to \\(a\\) at the end. A route that solves the problem is \\(a \\to b \\to c \\to d \\to e \\to f \\to b \\to g \\to c \\to e \\to g \\to f \\to a\\). Notice that the explorer passes through \\(b\\), \\(c\\), \\(e\\), \\(f\\), and \\(g\\) more than once; what matters is that each of the 12 roads (edges) of the graph is used exactly once.\nThe traveler problem. A traveler wants to visit each city exactly once, starting at \\(a\\) and returning to \\(a\\) at the end. A route that solves this problem is \\(a \\to b \\to c \\to d \\to e \\to g \\to f \\to a\\). Here the logic is reversed: each city (vertex) appears only once, but not every road (edge) needs to be used; the diagonal \\(bg\\), for example, is left out.\nIn short The explorer problem is to find a closed trail that includes every edge of the graph. The traveler problem is to find a cycle that includes every vertex of the graph. The first is the Eulerian problem; the second, the Hamiltonian one. We will treat them separately, starting with the simpler one.\nEulerian Graphs # Eulerian Trail, Eulerian Circuit, and Eulerian Graph An Eulerian trail in a graph \\(G\\) is a trail (with no repeated edges) that includes each edge of \\(G\\) exactly once. If this trail is closed (starts and ends at the same vertex), we call it an Eulerian circuit.\nA connected graph \\(G\\) is Eulerian if it admits an Eulerian circuit.\nEuler\u0026rsquo;s Theorem # The informal argument we gave in the opening (each pass through a region uses up a pair of bridges, so only the start and the end can have odd degree) is the easy half of the following theorem. Let us state and prove it precisely.\nEuler\u0026rsquo;s Theorem A connected graph \\(G\\) is Eulerian if and only if every vertex of \\(G\\) has even degree.\nProof of the necessary condition (Eulerian → even degrees):\nLet \\(C\\) be an Eulerian circuit of \\(G\\). Each time \\(C\\) passes through a vertex \\(v\\), it \u0026ldquo;enters\u0026rdquo; through one edge and \u0026ldquo;leaves\u0026rdquo; through another, using up two units of the degree of \\(v\\) (the starting vertex counts the exit at the beginning and the entry at the end, since the circuit is closed). Since each edge of the graph is used exactly once, the degree of \\(v\\) equals twice the number of passes of \\(C\\) through \\(v\\). Therefore, \\(d(v)\\) is even for every vertex \\(v\\) of \\(G\\), which is what we wanted to prove. \\(\\blacksquare\\)\nProof of the sufficient condition (even degrees → Eulerian): by strong induction on the number \\(m\\) of edges. Suppose that every connected graph with even degrees, at least one edge, and fewer than \\(m\\) edges is Eulerian, and let \\(G\\) be a connected graph with \\(m\\) edges (\\(m \\geq 1\\)) and all degrees even.\nStep 1: \\(G\\) has a cycle. Since \\(G\\) is connected and has edges, every vertex has degree at least 1; since the degrees are even, every vertex has degree at least 2. Take a path \\(P = v_0, v_1, \\ldots, v_k\\) of maximum length in \\(G\\). The vertex \\(v_k\\) has at least two neighbors: one is \\(v_{k-1}\\), and the other, \\(w\\), must be in \\(P\\). If it were not, we could extend \\(P\\) to \\(w\\), and \\(P\\) would not be maximum. If \\(w = v_i\\), then \\(v_i, v_{i+1}, \\ldots, v_k, v_i\\) is a cycle \\(C\\).\nStep 2: remove the cycle. Let \\(H\\) be the graph obtained by removing from \\(G\\) the edges of \\(C\\), without removing vertices. Each vertex of \\(C\\) loses exactly 2 from its degree, and the others lose nothing: all degrees of \\(H\\) remain even. \\(H\\) may be disconnected, but each connected component of \\(H\\) is a connected graph, with even degrees and fewer than \\(m\\) edges. By the inductive hypothesis, each component that has edges admits an Eulerian circuit.\nStep 3: stitch the circuits together. Every component of \\(H\\) with edges touches \\(C\\). Indeed, since \\(G\\) is connected, there is a path in \\(G\\) from a vertex of that component to \\(C\\); the first edge of that path that is not in \\(H\\) is an edge of \\(C\\), and the vertex where it starts belongs to both the component and \\(C\\). Now walk along \\(C\\): upon first arriving at a vertex of \\(C\\) where there is still an unvisited component of \\(H\\), take a detour along the Eulerian circuit of that component, which starts and ends at that vertex, and then continue along \\(C\\). The result is a closed trail that uses each edge of \\(C\\) and each edge of \\(H\\) exactly once, that is, each edge of \\(G\\) exactly once. (No separate base case is needed: if \\(H\\) has no edges, the circuit is just \\(C\\) itself.)\nTherefore, \\(G\\) is Eulerian; together with the necessary condition, a connected graph is Eulerian if and only if all its vertices have even degree, which is what we wanted to prove. \\(\\blacksquare\\)\nWhat about multigraphs? The theorem and both proofs also hold for multigraphs, counting each loop twice in the degree. The only difference is in Step 1: the \u0026ldquo;cycle\u0026rdquo; found may be a loop, or two parallel edges between the same two vertices. It is in this form that Euler\u0026rsquo;s Theorem is used in the Christofides-Serdyukov algorithm, later on.\nThis theorem is remarkable because it gives an efficient way to decide whether a graph is Eulerian: just check the parity of the degree of each vertex, without having to actually look for the circuit.\nExample: when graphs stop being Eulerian # Consider the four graphs below, all on subsets of the vertices \\(a, b, c, d, e, f, g\\); the first reuses the seven-city graph we saw in the motivation. The following figure shows the four side by side.\nG1 and G2 have all degrees even (Eulerian); G3 loses parity when the hexagon is replaced by a square with diagonals; G4 adds two pendant vertices of degree 1. \\(G_1\\): the seven-city graph Statement: \\(G_1\\) is the seven-city graph itself, with vertices \\(a, b, c, d, e, f, g\\). Is it Eulerian?\nSolution: Degrees: \\(d(a) = d(d) = 2\\); \\(d(b) = d(c) = d(e) = d(f) = d(g) = 4\\). All degrees are even, so, by Euler\u0026rsquo;s Theorem, \\(G_1\\) is Eulerian.\n\\(G_2\\): two triangular \u0026ldquo;wings\u0026rdquo; sharing a vertex Statement: Vertices \\(b, c, e, f, g\\); edges \\(bc, bg, cg, ef, eg, fg\\), two triangles, \\(bcg\\) and \\(efg\\), sharing the vertex \\(g\\). Is \\(G_2\\) Eulerian?\nSolution: Degrees: \\(d(b) = d(c) = d(e) = d(f) = 2\\); \\(d(g) = 4\\). All even, so \\(G_2\\) is Eulerian. One possible circuit is \\(b \\to c \\to g \\to f \\to e \\to g \\to b\\); notice that \\(g\\) is visited twice, which is perfectly valid in an Eulerian circuit, which does not repeat edges but may repeat vertices.\n\\(G_3\\): a square with both diagonals Statement: Vertices \\(b, c, e, f, g\\); edges \\(bc, ce, ef, fb, bg, cg, eg, fg\\). Is \\(G_3\\) Eulerian?\nSolution: Degrees: \\(d(b) = d(c) = d(e) = d(f) = 3\\); \\(d(g) = 4\\). Four vertices of odd degree: \\(G_3\\) is not Eulerian.\n\\(G_4\\): the square of \\(G_3\\) with two pendant vertices Statement: Vertices \\(a, b, c, d, e, f, g\\); the same edges as \\(G_3\\), plus \\(ab\\) and \\(cd\\). Is \\(G_4\\) Eulerian?\nSolution: Degrees: \\(d(a) = d(d) = 1\\); \\(d(b) = d(c) = d(g) = 4\\); \\(d(e) = d(f) = 3\\). Four vertices of odd degree (\\(a\\), \\(d\\), \\(e\\), \\(f\\)): \\(G_4\\) is not Eulerian.\nChecking with code Checking the parity of the degrees by hand works well for small graphs, but networkx already ships this check in nx.is_eulerian, and can even find an explicit Eulerian circuit with nx.eulerian_circuit, which implements Hierholzer\u0026rsquo;s algorithm:\nimport networkx as nx G1 = nx.Graph([(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;)]) G2 = nx.Graph([(\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;)]) G3 = nx.Graph([(\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;)]) G4 = nx.Graph([(\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;)]) for name, G in [(\u0026#34;G1\u0026#34;, G1), (\u0026#34;G2\u0026#34;, G2), (\u0026#34;G3\u0026#34;, G3), (\u0026#34;G4\u0026#34;, G4)]: print(f\u0026#34;Is {name} Eulerian? {nx.is_eulerian(G)}\u0026#34;) print(\u0026#34;Eulerian circuit of G1:\u0026#34;, list(nx.eulerian_circuit(G1, source=\u0026#34;a\u0026#34;))) Is G1 Eulerian? True Is G2 Eulerian? True Is G3 Eulerian? False Is G4 Eulerian? False Eulerian circuit of G1: [(\u0026#39;a\u0026#39;, \u0026#39;f\u0026#39;), (\u0026#39;f\u0026#39;, \u0026#39;g\u0026#39;), (\u0026#39;g\u0026#39;, \u0026#39;e\u0026#39;), (\u0026#39;e\u0026#39;, \u0026#39;c\u0026#39;), (\u0026#39;c\u0026#39;, \u0026#39;g\u0026#39;), (\u0026#39;g\u0026#39;, \u0026#39;b\u0026#39;), (\u0026#39;b\u0026#39;, \u0026#39;f\u0026#39;), (\u0026#39;f\u0026#39;, \u0026#39;e\u0026#39;), (\u0026#39;e\u0026#39;, \u0026#39;d\u0026#39;), (\u0026#39;d\u0026#39;, \u0026#39;c\u0026#39;), (\u0026#39;c\u0026#39;, \u0026#39;b\u0026#39;), (\u0026#39;b\u0026#39;, \u0026#39;a\u0026#39;)] The result confirms exactly the analysis we did by hand. The circuit found for \\(G_1\\) is just one among several possible; notice that it passes through \\(g\\) twice, which is valid, since the only requirement is not to repeat edges.\nOpen Eulerian trails # Not every graph with an Eulerian trail needs it to be closed. By relaxing the requirement of returning to the starting point, we arrive at an open version of the theorem, exactly the generalization we promised in Graphs: A Language for Modeling Connections, the article that introduced the Bridges of Königsberg problem and gave rise to all of graph theory.\nTheorem: Open Eulerian Trail A connected graph admits an open Eulerian trail (one that starts and ends at distinct vertices) if and only if it has exactly two vertices of odd degree. The trail starts at one of these two vertices and ends at the other.\nIt is worth quickly recalling the original problem, even for those who have not read the first article of the series: four regions of Königsberg (Altstadt, Kneiphof, Lomse, and Vorstadt), connected by seven bridges over the Pregel River, and the question was whether one could cross each bridge exactly once. The map below shows the layout of the bridges:\nHistorical map of the seven bridges of Königsberg over the Pregel River, in 18th-century Prussia. Source: Wikimedia Commons Euler abstracted this map into the graph below: each region became a vertex, each bridge an edge, and the double edges between A-B and B-D represent the pairs of bridges that existed between those regions.\nKönigsberg graph: A=Altstadt, B=Kneiphof, C=Lomse, D=Vorstadt. The double edges represent two bridges between those regions; the total is seven. Applying it to Königsberg The same four odd degrees of Königsberg that we saw in the opening are enough to conclude more than we already knew: since there are four vertices of odd degree (more than two), Königsberg admits neither a closed Eulerian circuit (Euler\u0026rsquo;s Theorem) nor an open Eulerian trail (this theorem). The answer to the original challenge is, therefore, a definitive no, whatever starting or ending point is chosen.\nHamiltonian Graphs # Hamiltonian Cycle and Hamiltonian Graph A Hamiltonian cycle in \\(G\\) is a cycle that includes each vertex of \\(G\\) exactly once. A graph \\(G\\) is Hamiltonian if it contains a Hamiltonian cycle.\nThe difference between Eulerian and Hamiltonian Eulerian: traverses each edge exactly once (may repeat vertices). Hamiltonian: traverses each vertex exactly once (may leave edges out). Hamilton and the Icosian Game # The name \u0026ldquo;Hamiltonian\u0026rdquo; comes from Sir William Rowan Hamilton (1805-1865), who popularized the problem with the Icosian Game, a puzzle presented in 1857 and sold from 1859. The board represents the graph of a dodecahedron (20 vertices, 30 edges, each vertex with exactly 3 neighbors) and the challenge is: given an initial sequence of five consecutive vertices of a route, complete a cycle that visits the remaining 15 vertices exactly once and returns to the start. The figure below shows the graph drawn as an outer pentagon, a band of 10 vertices, and an inner pentagon (the classic layout used on the original board), with the 5 initial vertices of the challenge in green and a complete Hamiltonian cycle highlighted in orange.\nThe dodecahedron graph drawn as an outer pentagon, a band of 10 vertices, and an inner pentagon. In green, the 5 initial vertices of the challenge; in orange, a complete Hamiltonian cycle. Despite the name, the game\u0026rsquo;s graph is that of the dodecahedron, not of the icosahedron. The prefix comes from the Greek for \u0026ldquo;twenty\u0026rdquo;: the dodecahedron has 20 vertices, and a cycle that goes through all of them has 20 edges.\nNot every question has an easy answer # Unlike the Eulerian problem, deciding whether a graph is Hamiltonian seems, at first sight, a similar question, but no complete and efficient characterization like Euler\u0026rsquo;s Theorem is known. Deciding whether an arbitrary graph is Hamiltonian is, in fact, an NP-complete problem (the acronym stands for Nondeterministic Polynomial time and identifies the class of the hardest decision problems among those verifiable in polynomial time): no efficient algorithm is known that solves the general case. What exist are partial conditions: sufficient, but not necessary.\nA simple observation helps build intuition: the more edges a graph has, the greater the chance that it is Hamiltonian. The cycle \\(C_n\\) (with \\(n\\) vertices) is already, trivially, Hamiltonian: it is itself the Hamiltonian cycle. If we add edges to \\(C_n\\), the resulting graph still contains that same cycle as a subgraph, so it remains Hamiltonian. In the limit, the complete graph \\(K_n\\), which has all possible edges, is also Hamiltonian. The sequence of figures below illustrates this progression.\nThe cycle C5 (orange) remains a valid Hamiltonian cycle in all four graphs, even as chords (gray) are added until reaching K5. This observation is not a proof, only an intuition that \u0026ldquo;dense\u0026rdquo; graphs (with many edges) tend to be Hamiltonian. But it motivates exactly the two most important sufficient-condition theorems of the theory, published a few years apart.\nSufficient Conditions: Dirac and Ore # Dirac\u0026rsquo;s Theorem (1952) If \\(G\\) has \\(n \\geq 3\\) vertices and \\(d(v) \\geq \\dfrac{n}{2}\\) for every \\(v \\in V(G)\\), then \\(G\\) is Hamiltonian.\nOre\u0026rsquo;s Theorem (1960) If \\(G\\) has \\(n \\geq 3\\) vertices and \\(d(v) + d(w) \\geq n\\) for every pair of nonadjacent vertices \\(v, w\\), then \\(G\\) is Hamiltonian.\nNote that Dirac\u0026rsquo;s Theorem is a special case of Ore\u0026rsquo;s: if \\(d(v) \\geq n/2\\) and \\(d(w) \\geq n/2\\) for any \\(v, w\\), then \\(d(v) + d(w) \\geq n\\) even when \\(v\\) and \\(w\\) are not adjacent. Ore\u0026rsquo;s is, therefore, a strictly broader condition. The two graphs below show a case where Dirac already settles the question and another where only Ore does.\nOn the left, the triangular prism G1, 3-regular, satisfies Dirac. On the right, G2 has a vertex b of degree 2 (highlighted) that breaks Dirac\u0026rsquo;s condition, but Ore still guarantees Hamiltonicity. In orange, a Hamiltonian cycle of each graph. Example: the triangular prism satisfies Dirac Statement: The graph \\(G_1\\) has 6 vertices \\(a, b, c, d, e, f\\) with edges \\(ab, ad, ae, bc, bf, cd, cf, de, ef\\), a triangular prism: two triangles, \\(ade\\) and \\(bcf\\), joined by the three edges \\(ab\\), \\(cd\\), and \\(ef\\). Is \\(G_1\\) Hamiltonian?\nSolution: \\(G_1\\) is 3-regular: \\(d(v) = 3\\) for every vertex. Since \\(n = 6\\), we have \\(n/2 = 3\\), so \\(d(v) = 3 \\geq 3 = n/2\\) for every \\(v\\). By Dirac\u0026rsquo;s Theorem, \\(G_1\\) is Hamiltonian; indeed, \\(a \\to d \\to c \\to b \\to f \\to e \\to a\\) is a valid Hamiltonian cycle.\nExample: when Dirac fails but Ore works Statement: The graph \\(G_2\\) has vertices \\(a, b, c, d, e\\) and edges \\(ab, bc, ae, ad, ce, cd, ed\\). Is \\(G_2\\) Hamiltonian?\nSolution: Degrees: \\(d(b) = 2\\); \\(d(a) = d(c) = d(d) = d(e) = 3\\). Since \\(n = 5\\), Dirac\u0026rsquo;s condition requires degree \\(\\geq 2.5\\) for every vertex, but \\(d(b) = 2 \u0026lt; 2.5\\). Dirac\u0026rsquo;s Theorem does not apply.\nTesting Ore on the three pairs of nonadjacent vertices (\\(a,c\\)), (\\(b,d\\)), and (\\(b,e\\)):\n\\(d(a) + d(c) = 3 + 3 = 6 \\geq 5\\) ✓ \\(d(b) + d(d) = 2 + 3 = 5 \\geq 5\\) ✓ \\(d(b) + d(e) = 2 + 3 = 5 \\geq 5\\) ✓ All nonadjacent pairs satisfy the condition: by Ore\u0026rsquo;s Theorem, \\(G_2\\) is Hamiltonian, even though Dirac guarantees nothing. An explicit Hamiltonian cycle: \\(a \\to b \\to c \\to d \\to e \\to a\\). This example shows Ore being strictly stronger (broader) than Dirac.\nSufficient conditions, not necessary There are Hamiltonian graphs that satisfy neither Dirac nor Ore. The cycle \\(C_5\\) is Hamiltonian (it is itself the Hamiltonian cycle), but \\(d(v) = 2\\) for every vertex.\nDirac: it would need \\(d(v) \\geq n/2 = 2.5\\); we have only \\(d(v) = 2\\). Fails. Ore: every pair of nonadjacent vertices in \\(C_5\\) has \\(d(v) + d(w) = 2 + 2 = 4 \u0026lt; 5 = n\\). Fails too. Both conditions are only sufficient: their absence does not allow us to conclude anything about Hamiltonicity.\nThe Traveling Salesman Problem (TSP) # The Traveling Salesman Problem (TSP) is one of the most famous questions in mathematics and computer science:\nTraveling Salesman Problem Given a weighted graph (each edge with a weight, representing distance or cost), find the Hamiltonian cycle of minimum total cost.\nA bit of history # The problem is older than it looks. A German manual from 1832 for traveling salesmen already mentioned the importance of planning the route well, with no mathematical treatment. Mathematics entered the scene still in the 19th century, with William Rowan Hamilton and Thomas Kirkman, who studied cycles that visit each vertex of a graph, the same concept that motivated the Icosian Game we saw earlier. The general form of the problem appeared in the 1930s: the Austrian mathematician Karl Menger, then in Vienna and later at Harvard, studied it under the name Botenproblem (\u0026ldquo;messenger problem\u0026rdquo;). The name \u0026ldquo;traveling salesman problem\u0026rdquo; first appears in a 1949 report by the mathematician Julia Robinson for the RAND Corporation.\nThe first major practical breakthrough came in 1954: George Dantzig, Delbert Fulkerson, and Selmer Johnson optimally solved an instance with 49 cities using a cutting-plane method, the seed of the combinatorial optimization techniques used to this day.\nA small example # Consider five cities \\(A, B, C, D, E\\) connected by the roads and distances of the graph below.\nFive cities A, B, C, D, E and the distances between them. In orange, the minimum-cost Hamiltonian cycle: A→C→B→D→E→A, totaling 14. Two possible Hamiltonian cycles:\n\\(C_1: A \\to B \\to C \\to D \\to E \\to A\\), with cost \\(5+2+4+3+4 = 18\\) \\(C_2: A \\to C \\to B \\to D \\to E \\to A\\), with cost \\(3+2+2+3+4 = 14\\) Testing all possible Hamiltonian cycles of this graph, \\(C_2\\) is indeed the one with minimum cost, the optimal solution of this small example.\nWhy it is hard # With \\(n\\) cities, starting from a fixed city, the number of Hamiltonian cycles we can form (counting the two directions of travel as different) is \\((n-1)!\\):\n10 cities: \\(9! = 362{,}880\\) routes 20 cities: \\(19! \\approx 1.22 \\times 10^{17}\\) routes 50 cities: checking by brute force would take longer than the age of the universe (Since traversing a cycle in one direction or the other gives exactly the same total cost, some authors divide this number by 2 to count only distinct cycles. The order of magnitude of the problem does not change.)\nThe animation below shows brute force exhaustively testing the cycles of a 7-city TSP until it finds the optimum:\nBrute force testing all (7-1)!/2 = 360 possible cycles of a 7-city TSP, until it finds the optimum. Source: Wikimedia Commons It is worth being precise about what exactly is hard here. The decision version of the problem (\u0026ldquo;is there a Hamiltonian cycle of cost at most \\(k\\)?\u0026rdquo;) is NP-complete. The optimization version (\u0026ldquo;what is the minimum-cost cycle?\u0026rdquo;, which is the one we really care about) is NP-hard: at least as hard as any problem in NP. In 1972, Richard Karp proved that deciding whether a graph has a Hamiltonian cycle (the problem we studied in this very article) is NP-complete. That is where the difficulty of the TSP comes from: giving weight 1 to the edges of a graph with \\(n\\) vertices and weight 2 to the pairs of vertices that are not edges, the graph has a Hamiltonian cycle if and only if the TSP has a cycle of cost \\(n\\), one more link between the two central themes of this text.\nExact algorithms: beyond brute force # Brute force is not the only exact option. The Held-Karp algorithm (1962) uses dynamic programming to solve the TSP in \\(O(n^2 2^n)\\) time, still exponential, but much better than \\(O(n!)\\):\n\\(n\\) \\((n-1)!\\) (brute force) \\(n^2 2^n\\) (Held-Karp) 10 362,880 102,400 20 \\(1.22 \\times 10^{17}\\) \\(4.19 \\times 10^{8}\\) 30 \\(8.84 \\times 10^{30}\\) \\(9.66 \\times 10^{11}\\) The idea: for each subset \\(S\\) of cities already visited and each city \\(j \\in S\\), we store the minimum cost to reach \\(j\\) having visited exactly the cities in \\(S\\). Each new subset is built from smaller subsets, avoiding recomputing the same partial path several times, the hallmark of any dynamic programming.\nThe code below implements Held-Karp for our five-city example (using a bitmask to represent subsets of cities, in which bit \\(j\\) of an integer indicates whether city \\(j\\) has already been visited):\nimport math from itertools import combinations cities = [\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;] INF = math.inf dist = { (\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;): 5, (\u0026#34;A\u0026#34;, \u0026#34;C\u0026#34;): 3, (\u0026#34;A\u0026#34;, \u0026#34;D\u0026#34;): 6, (\u0026#34;A\u0026#34;, \u0026#34;E\u0026#34;): 4, (\u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;): 2, (\u0026#34;B\u0026#34;, \u0026#34;D\u0026#34;): 2, (\u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;): 4, (\u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;): 3, } def d(i, j): if i == j: return 0 return dist.get((i, j), dist.get((j, i), INF)) n = len(cities) # dp[(subset, j)] = minimum cost to visit exactly `subset` # (bitmask over cities 1..n-1), ending at city j dp = {} for j in range(1, n): dp[(1 \u0026lt;\u0026lt; j, j)] = d(cities[0], cities[j]) for size in range(2, n): for subset in combinations(range(1, n), size): bits = sum(1 \u0026lt;\u0026lt; b for b in subset) for j in subset: bits_without_j = bits \u0026amp; ~(1 \u0026lt;\u0026lt; j) dp[(bits, j)] = min( dp[(bits_without_j, k)] + d(cities[k], cities[j]) for k in subset if k != j ) all_cities = sum(1 \u0026lt;\u0026lt; b for b in range(1, n)) optimal_cost = min(dp[(all_cities, j)] + d(cities[j], cities[0]) for j in range(1, n)) print(f\u0026#34;Optimal cycle cost: {optimal_cost}\u0026#34;) Optimal cycle cost: 14 The result matches exactly the \\(C_2\\) we found by inspection. The difference is that Held-Karp finds this value systematically, without having to enumerate the cycles manually, and keeps working (in principle) for larger instances, although \\(2^n\\) still grows too fast to be practical beyond about 20-30 cities.\nFor larger instances, the method of choice is branch-and-bound combined with linear programming techniques (branch-and-cut), which discards whole branches of the search tree whenever it can prove that they do not lead to a solution better than the best one known. The animation below shows the same 7-city TSP solved this way, exploring far fewer branches than brute force:\nBranch-and-bound explores far fewer branches than brute force, discarding early those that cannot improve the current solution. Source: Wikimedia Commons It is this family of techniques (refined over decades in the Concorde TSP Solver software) that made it possible to optimally solve huge real instances: 15,112 cities in Germany in 2001, 24,978 cities in Sweden in 2004, and a record of 85,900 points (a drilling problem for printed circuit boards) in 2006, using more than 136 CPU-years of processing.\nHeuristics: good solutions, quickly # When the exact approach is too expensive, heuristics find good (not necessarily optimal) solutions in much less time.\nThe simplest is nearest neighbor: start at any city and, at each step, go to the nearest unvisited city. See below this heuristic in action on a 7-city TSP:\nNearest neighbor produces different solutions depending on the starting city, and offers no optimality guarantee. Source: Wikimedia Commons import math cities = [\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;] INF = math.inf dist = { (\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;): 5, (\u0026#34;A\u0026#34;, \u0026#34;C\u0026#34;): 3, (\u0026#34;A\u0026#34;, \u0026#34;D\u0026#34;): 6, (\u0026#34;A\u0026#34;, \u0026#34;E\u0026#34;): 4, (\u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;): 2, (\u0026#34;B\u0026#34;, \u0026#34;D\u0026#34;): 2, (\u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;): 4, (\u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;): 3, } def d(i, j): if i == j: return 0 return dist.get((i, j), dist.get((j, i), INF)) def nearest_neighbor(start): unvisited = set(cities) - {start} tour = [start] current = start total_cost = 0 while unvisited: # distance tie: the first city in alphabetical order stays next_city = min(sorted(unvisited), key=lambda c: d(current, c)) total_cost += d(current, next_city) tour.append(next_city) unvisited.remove(next_city) current = next_city total_cost += d(current, start) tour.append(start) return tour, total_cost for city in cities: tour, cost = nearest_neighbor(city) print(f\u0026#34;{\u0026#39; -\u0026gt; \u0026#39;.join(tour)}: cost {cost}\u0026#34;) A -\u0026gt; C -\u0026gt; B -\u0026gt; D -\u0026gt; E -\u0026gt; A: cost 14 B -\u0026gt; C -\u0026gt; A -\u0026gt; E -\u0026gt; D -\u0026gt; B: cost 14 C -\u0026gt; B -\u0026gt; D -\u0026gt; E -\u0026gt; A -\u0026gt; C: cost 14 D -\u0026gt; B -\u0026gt; C -\u0026gt; A -\u0026gt; E -\u0026gt; D: cost 14 E -\u0026gt; D -\u0026gt; B -\u0026gt; C -\u0026gt; A -\u0026gt; E: cost 14 Starting from B, cities C and D are at the same distance (2), and the code breaks the tie by alphabetical order: it goes to C. With another tiebreaking criterion, the route would come out different, and that is how it is with the heuristic in general: the result depends on how ties are broken.\nIn this small example, nearest neighbor finds the optimum (14) from any starting city, but this is a coincidence of the reduced example, not a general guarantee. On larger instances, this heuristic typically produces tours about 25% longer than the optimum, and in the worst case it can be arbitrarily worse. Still, its simplicity makes it a good starting point for more refined heuristics.\nOne such improvement is 2-opt: given any tour, look for two edges that, if swapped (undoing a crossing), reduce the total distance. Repeat until no more improvements are found. The figure below shows one such swap:\nSwapping two crossing edges for two others, without crossing, shortens the tour. Source: Wikimedia Commons Applied repeatedly on an initial solution (such as the one from nearest neighbor), 2-opt usually improves the tour considerably: on Euclidean instances, it gives on average solutions about 5% better than those of the Christofides-Serdyukov algorithm, which we will see next, and more sophisticated variants, such as 3-opt and the Lin-Kernighan heuristic, get even closer to the optimum.\nThe Christofides-Serdyukov algorithm: Eulerian and Hamiltonian together # No heuristic illustrates the connection between the two themes of this article better than the Christofides-Serdyukov algorithm (1976). It works on metric graphs (where the distances satisfy the triangle inequality) and guarantees a tour with cost at most 1.5 times the optimum. For decades, nobody got a better guarantee. In 2011 came a tiny improvement, but only for a particular case, the graphic TSP (in which the distance between two cities is the number of edges of the shortest path in an unweighted graph); for the general metric TSP, the factor 1.5 was only beaten in 2020, by Anna Karlin, Nathan Klein, and Shayan Oveis Gharan, with a randomized algorithm of factor \\(1.5 - 10^{-36}\\) (see the entry Christofides algorithm).\nThe steps of the algorithm use, in sequence, a minimum spanning tree, a minimum-weight matching, and, here is the connection, an Eulerian circuit:\n1 Minimum spanning tree Build the minimum spanning tree of the weighted graph. 2 Matching on the odd-degree vertices In the minimum spanning tree, identify the vertices of odd degree. By the Handshaking Lemma, this set always has an even number of vertices, which makes it possible to pair them up. Compute the minimum-weight matching among them and add these edges to the tree. 3 Eulerian circuit Each odd-degree vertex identified in the previous step had its degree increased by exactly 1 by the matching, and therefore every vertex of the resulting multigraph (tree plus matching) now has even degree. By Euler\u0026rsquo;s Theorem that we proved in this article, in the version for multigraphs, this multigraph is Eulerian: find an Eulerian circuit in it. 4 Shortcut to a Hamiltonian cycle Traverse the Eulerian circuit in the order found, but skip any city already visited, going straight to the next new city. Since the graph is metric, \u0026ldquo;cutting corners\u0026rdquo; this way never increases the total cost (triangle inequality). The result is a Hamiltonian cycle, the answer of the algorithm. The figure below illustrates step 2, the matching of the odd-degree vertices:\nThe odd-degree vertices of the minimum spanning tree (in red) are matched by minimum-weight edges (dashed). Source: Wikimedia Commons And this one shows the shortcut of step 4, which turns the Eulerian circuit into a Hamiltonian cycle:\nThe Eulerian circuit of the multigraph (tree + matching) is traversed skipping repeated vertices, producing a Hamiltonian cycle. Source: Wikimedia Commons Checking with code networkx already implements the four steps above ready to use, in nx.approximation.christofides. The only requirement is that the graph be complete. Since our five-city example does not have the edges \\(EB\\) and \\(EC\\), we complete the graph with the metric closure computed by hand (the shortest possible distance between these pairs going through other cities): \\(d(E,B) = d(E,D) + d(D,B) = 3 + 2 = 5\\) and \\(d(E,C) = d(E,A) + d(A,C) = 4 + 3 = 7\\).\nimport networkx as nx G_complete = nx.Graph() G_complete.add_weighted_edges_from([ (\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, 5), (\u0026#34;A\u0026#34;, \u0026#34;C\u0026#34;, 3), (\u0026#34;A\u0026#34;, \u0026#34;D\u0026#34;, 6), (\u0026#34;A\u0026#34;, \u0026#34;E\u0026#34;, 4), (\u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;, 2), (\u0026#34;B\u0026#34;, \u0026#34;D\u0026#34;, 2), (\u0026#34;B\u0026#34;, \u0026#34;E\u0026#34;, 5), (\u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, 4), (\u0026#34;C\u0026#34;, \u0026#34;E\u0026#34;, 7), (\u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;, 3), ]) tour = nx.approximation.christofides(G_complete, weight=\u0026#34;weight\u0026#34;) cost = sum(G_complete[tour[i]][tour[i + 1]][\u0026#34;weight\u0026#34;] for i in range(len(tour) - 1)) print(\u0026#34;Tour:\u0026#34;, \u0026#34; -\u0026gt; \u0026#34;.join(tour)) print(\u0026#34;Cost:\u0026#34;, cost) Tour: A -\u0026gt; E -\u0026gt; D -\u0026gt; B -\u0026gt; C -\u0026gt; A Cost: 14 In this small example, Christofides-Serdyukov finds the optimum itself (14), the same value as \\(C_2\\), only traversed in the opposite direction. This is not guaranteed in general: the 1.5x guarantee is a worst-case bound, not a typical prediction. On larger, more irregular instances, the algorithm usually delivers tours much closer to the optimum than this theoretical bound suggests.\nIn other words: Christofides-Serdyukov deliberately builds an Eulerian graph just to \u0026ldquo;shortcut\u0026rdquo; it into a Hamiltonian graph, using the two theorems that are the subject of this article as tools of a real approximation algorithm, still used today.\nOther popular metaheuristics, such as genetic algorithms, simulated annealing, tabu search, and ant colony optimization (which simulates ants depositing pheromone on the shortest paths), also produce good practical results, but without the mathematical 1.5x guarantee that Christofides-Serdyukov offers for metric graphs.\nSpecial cases # Not every TSP is equally hard in practice:\nMetric TSP: when the distances satisfy the triangle inequality (\\(d(u,w) \\leq d(u,v) + d(v,w)\\) for any \\(u, v, w\\)), the most common case in real applications, since geographic distances behave this way. It is what enables the 1.5x guarantee of Christofides-Serdyukov. Euclidean TSP: when the cities are points in the plane and the distance is the usual Euclidean distance. It remains NP-hard, but admits a PTAS (polynomial-time approximation scheme) discovered by Sanjeev Arora and Joseph Mitchell (work awarded the 2010 Gödel Prize), which comes as close to the optimum as desired, in polynomial time for any fixed precision. These special cases explain why the TSP, although NP-hard in general, is routinely solved in practice: most real applications fall into one of these more tractable cases.\nReal applications:\nLogistics and delivery routing Printed circuit board design (minimizing the soldering probe path) Sequencing of operations on industrial machines Route planning for robots in automated warehouses DNA sequencing in bioinformatics Telescope positioning in astronomical surveys For the complete mathematical treatment, including asymmetric variants, integer linear programming formulations, and the most recent computational records, it is worth consulting the Wikipedia article on the TSP.\nSummary # Result Statement Euler\u0026rsquo;s Theorem connected \\(G\\) is Eulerian \\(\\Leftrightarrow\\) every vertex has even degree Open Eulerian Trail connected \\(G\\) admits an open Eulerian trail \\(\\Leftrightarrow\\) exactly 2 vertices of odd degree Dirac\u0026rsquo;s Theorem \\(d(v) \\geq n/2\\) for all \\(v\\) \\(\\Rightarrow\\) \\(G\\) is Hamiltonian Ore\u0026rsquo;s Theorem \\(d(v)+d(w) \\geq n\\) for every nonadjacent pair \\(\\Rightarrow\\) \\(G\\) is Hamiltonian Traveling Salesman Problem find, among the Hamiltonian cycles of a weighted graph, the one with minimum total cost Christofides-Serdyukov on metric graphs, builds an Eulerian circuit (minimum spanning tree + matching) and shortcuts it into a Hamiltonian cycle of cost \\(\\leq 1.5\\) times the optimum Counting Hamiltonian cycles with \\(n\\) cities, there are \\((n-1)!\\) possible routes (brute force); Held-Karp solves it in \\(O(n^2 2^n)\\) Comparison: Eulerian vs. Hamiltonian # Property Eulerian Hamiltonian Traverses All edges All vertices Exact criterion Yes: all degrees even No general criterion exists Open version Exactly 2 odd degrees - Sufficient condition - Dirac or Ore Verification complexity \\(O(n + m)\\) (degrees and connectivity) NP-complete Optimization version - TSP (NP-hard) Exercises # Exercise 1: Eulerian or not? Determine whether the following graphs are Eulerian, have an open Eulerian trail, or neither:\n(a) \\(K_5\\) (b) \\(K_6\\) (c) \\(K_{3,3}\\) (d) \\(C_7\\)\nSolutions (a) \\(K_5\\): all 5 vertices have degree 4 (even). Eulerian. ✓\n(b) \\(K_6\\): all 6 vertices have degree 5 (odd). There are 6 vertices of odd degree. No Eulerian trail (neither open nor closed).\n(c) \\(K_{3,3}\\): in complete bipartite graphs \\(K_{p,q}\\), each vertex in \\(X\\) has degree \\(q\\) and each vertex in \\(Y\\) has degree \\(p\\). In \\(K_{3,3}\\): all have degree 3 (odd). There are 6 vertices of odd degree. No Eulerian trail.\n(d) \\(C_7\\): 2-regular graph (all degrees even). Eulerian. The cycle itself is the Eulerian circuit.\nExercise 2: Applying Dirac Is the graph \\(K_4\\) (4 vertices, all with degree 3) Hamiltonian? Apply Dirac\u0026rsquo;s Theorem.\nSolution For \\(n = 4\\), Dirac\u0026rsquo;s condition requires \\(d(v) \\geq n/2 = 2\\). Since \\(d(v) = 3 \\geq 2\\) for every \\(v\\), Dirac\u0026rsquo;s Theorem confirms that \\(K_4\\) is Hamiltonian. ✓ (In fact, any ordering of the 4 vertices forms a Hamiltonian cycle in \\(K_4\\).)\nExercise 3: The Icosian Game challenge In the dodecahedron graph (20 vertices, each with exactly 3 neighbors, numbered 1 to 20 in the Icosian Game figure above), vertices \\(1\\) to \\(5\\) are exactly the outer pentagon of the drawing. A Hamiltonian cycle starts with the sequence \\(1 \\to 2 \\to 3 \\to 4 \\to 5\\). Complete this cycle by going through the remaining 15 vertices. Then, try to find a second solution, different from the first, that also starts with \\(1,2,3,4,5\\).\nSolution A first solution simply follows the numbering in order: \\(1 \\to 2 \\to 3 \\to \\cdots \\to 19 \\to 20 \\to 1\\).\nA second solution, distinct from the first but with the same beginning, is: \\(1 \\to 2 \\to 3 \\to 4 \\to 5 \\to 6 \\to 19 \\to 18 \\to 14 \\to 15 \\to 16 \\to 17 \\to 7 \\to 8 \\to 9 \\to 10 \\to 11 \\to 12 \\to 13 \\to 20 \\to 1\\).\nBoth use the same first five vertices, but diverge completely afterward, exactly the kind of challenge that made Hamilton\u0026rsquo;s Icosian Game fun (and far from trivial) for Victorian players.\nNext steps # Everything in this article dealt with traversals: which sequences of edges or vertices a graph allows us to traverse. At no point did we ask how that graph can be drawn: whether we can put it on paper without the edges crossing. It is exactly this limitation that closes this stage of the series: knowing that a graph is Eulerian or Hamiltonian says nothing about its geometry. In the next article, we investigate planar graphs: when it is possible to draw a graph in the plane without edge crossings, and what this has to do with the Four Color Theorem that we saw at the beginning of the series.\n","date":"2026-08-20","externalUrl":null,"permalink":"/2026/08/eulerian-hamiltonian-graphs/","section":"Blog","summary":"Traversing each edge exactly once is easy to decide: just look at the parity of the degrees (Euler’s Theorem). Deciding whether we can visit each vertex exactly once is NP-complete. We will see both problems, the theorems of Dirac and Ore, and the Traveling Salesman, from Held-Karp to Christofides.","title":"Eulerian and Hamiltonian Graphs","type":"posts"},{"content":"","date":"2026-08-18","externalUrl":null,"permalink":"/tags/arvores/","section":"tags","summary":"","title":"árvores","type":"tags"},{"content":"","date":"2026-08-18","externalUrl":null,"permalink":"/tags/trees/","section":"tags","summary":"","title":"trees","type":"tags"},{"content":" If you have ever used a file system, a binary search data structure, a heap, or parsed the syntax of a program, you have already worked with trees. They are among the most widely used data structures in computer science.\nWithin graph theory, trees also hold a special place: they are the simplest family of graphs and the one with the friendliest properties. That is why, whenever we want to prove a general result about graphs, the most natural strategy is to check first whether it holds for trees. If it fails even in this simplest case, there is no hope for the general case; if it holds, a good part of the proof work is already done. Trees also show up outside computing, describing everything from the structure of molecules in chemistry to syntactic analysis in linguistics, but it is in computing that they are most useful.\nWhy Do Trees Matter? # Trees are, at the same time, the simplest theoretical model in graph theory and one of the most widely used data structures in practice:\nFile systems: folders and files naturally form a rooted tree, where each folder is the child of exactly one parent folder. Compilers and interpreters: parsing a program produces an abstract syntax tree (AST), used to generate code or to interpret the language. Efficient search structures: binary search trees and heaps organize data hierarchically to make search, insertion, and removal fast, as long as the tree stays balanced, with small height. Minimum-cost networks: finding the minimum spanning tree of a network (the cheapest way to keep everything connected, with no redundant cycles) is a classic optimization problem in telecommunications and logistics. Definition of a Tree # Tree A tree is a connected and acyclic (cycle-free) graph.\nForest An acyclic graph is a forest: each connected component of a forest is a tree.\nThe definition of a tree is minimalist: a tree has the minimum number of edges needed to keep it connected, and removing any edge disconnects it.\nThe following three trees illustrate this definition well:\nThree trees: connected and acyclic The example above shows three trees that are quite different from one another (one looks like a crooked path, another is a star, the third branches into several levels), but all of them share the two properties that define the category: they are connected and have no cycles.\nA forest, in turn, is what is left when we give up connectivity but keep the absence of cycles: each isolated piece (each connected component) is, by itself, a tree. The same three trees above, for example, can be seen as the three connected components of a single forest:\nForest G with connected components G1, G2, and G3 Leaves # Leaf A leaf (or pendant vertex) is a vertex \\(v\\) such that \\(d(v) = 1\\).\nIn the same three trees, the leaves (highlighted in red) are exactly the vertices of degree 1, at the tips of the branches:\nLeaves (degree-1 vertices) in red A Characterization of Trees # The following result connects the definition of a tree (connected and acyclic) to an equivalent property, but one stated in a different way, in terms of paths:\nTheorem 1: Unique path A graph \\(G\\) is a tree if and only if there is a unique path between every pair of vertices of \\(G\\).\nProof:\n(⇒) Let \\(G\\) be a tree. By definition \\(G\\) is connected, so between each pair of vertices \\(v\\) and \\(w\\) there is at least one path. It remains to show that this path is unique.\nSuppose, for contradiction, that there are two distinct paths \\(P_1\\) and \\(P_2\\) between \\(v\\) and \\(w\\). Since the paths are distinct but share the endpoints \\(v\\) and \\(w\\), there are necessarily two vertices \\(t_1\\) and \\(t_2\\) (possibly \\(t_1 = v\\) and \\(t_2 = w\\), if the paths touch only at the endpoints) such that, between \\(t_1\\) and \\(t_2\\), the subpaths of \\(P_1\\) and \\(P_2\\) are vertex-disjoint, that is, \\(t_1\\) and \\(t_2\\) are the first point where the paths separate and the next point where they meet again.\nThe subpath of \\(P_1\\) between \\(t_1\\) and \\(t_2\\), joined to the subpath of \\(P_2\\) between \\(t_1\\) and \\(t_2\\) (traversed in the opposite direction), forms a cycle. This contradicts the fact that \\(G\\) is acyclic. Hence the path between \\(v\\) and \\(w\\) is unique.\n(⇐) Suppose there is a unique path between each pair of vertices of \\(G\\). In particular, there is at least one path between each pair of vertices, so \\(G\\) is connected.\nIt remains to show that \\(G\\) is acyclic. Suppose, for contradiction, that \\(C\\) is a cycle of \\(G\\), and let \\((v, w)\\) be an edge of \\(C\\). Then there are two distinct paths between \\(v\\) and \\(w\\): the edge \\((v,w)\\) itself and the path \\(C - (v,w)\\) (the rest of the cycle). This contradicts the unique-path hypothesis. Hence \\(G\\) is acyclic.\nSince \\(G\\) is connected and acyclic, \\(G\\) is a tree. Therefore, a graph is a tree if and only if there is a unique path between each pair of its vertices, which is what we wanted to prove. \\(\\blacksquare\\)\nIn the tree below, for example, there is a unique path between vertices \\(a\\) and \\(f\\), highlighted in red:\nUnique path between a and f Trees and the Number of Edges # Theorem 2: The relation m = n − 1 If \\(G\\) is a tree with \\(n\\) vertices, then \\(G\\) has exactly \\(n - 1\\) edges.\nProof by strong induction on \\(n\\):\nBase case: \\(n = 1\\). A tree with 1 vertex (the trivial graph) has 0 edges. \\(n - 1 = 0\\). ✓\nInductive hypothesis: suppose the result holds for every tree with fewer than \\(n\\) vertices (\\(n \\geq 1\\)).\nInductive step: let \\(T\\) be a tree with \\(n\\) vertices (\\(n \u0026gt; 1\\)). Since \\(n \u0026gt; 1\\), \\(T\\) contains at least one edge \\(e = (v, w)\\). Let \\(T^{\\prime}\\) be the graph obtained by removing \\(e\\) from \\(T\\) (without removing the vertices).\nBy Theorem 1, the unique path between \\(v\\) and \\(w\\) in \\(T\\) is the edge \\(e\\) itself. When \\(e\\) is removed, \\(v\\) and \\(w\\) are no longer connected in \\(T^{\\prime}\\): they end up in two distinct connected components of \\(T^{\\prime}\\). And there are no other components: every vertex \\(x\\) remains linked to \\(v\\) or to \\(w\\), because the path from \\(x\\) to \\(v\\) in \\(T\\), if it uses the edge \\(e\\), passes through \\(w\\) right before it. Let \\(T_1\\) and \\(T_2\\) be these two components. Each of them is acyclic (since it is a subgraph of a tree) and connected (it is a connected component); hence \\(T_1\\) and \\(T_2\\) are trees, each with fewer than \\(n\\) vertices.\nBy the inductive hypothesis, applied to \\(T_1\\) and \\(T_2\\): $$|E(T_1)| = |V(T_1)| - 1 \\qquad \\text{and} \\qquad |E(T_2)| = |V(T_2)| - 1$$Since \\(V(T_1) \\cup V(T_2) = V(T)\\) and \\(E(T_1) \\cup E(T_2) \\cup \\{e\\} = E(T)\\), we have: $$n = |V(T)| = |V(T_1)| + |V(T_2)|$$ $$m = |E(T)| = |E(T_1)| + |E(T_2)| + 1 = \\big(|V(T_1)| - 1\\big) + \\big(|V(T_2)| - 1\\big) + 1 = |V(T_1)| + |V(T_2)| - 1 = n - 1$$Therefore, by induction, every tree with \\(n\\) vertices has \\(m = n - 1\\) edges, which is what we wanted to prove. \\(\\blacksquare\\)\nChecking with code The proof above holds for any \\(n\\), but there is nothing like generating random trees of very different sizes and checking the relation \\(m = n-1\\) on each one:\nimport networkx as nx for n in [5, 10, 20, 50, 100]: T = nx.random_labeled_tree(n, seed=42) m, v = T.number_of_edges(), T.number_of_nodes() print(f\u0026#34;n={v:\u0026gt;3}: |V|={v:\u0026gt;3}, |E|={m:\u0026gt;3}, |E| == |V|-1? {m == v - 1}\u0026#34;) n= 5: |V|= 5, |E|= 4, |E| == |V|-1? True n= 10: |V|= 10, |E|= 9, |E| == |V|-1? True n= 20: |V|= 20, |E|= 19, |E| == |V|-1? True n= 50: |V|= 50, |E|= 49, |E| == |V|-1? True n=100: |V|=100, |E|= 99, |E| == |V|-1? True nx.random_labeled_tree(n) draws a random tree from among all \\(n^{n-2}\\) possible labeled trees with \\(n\\) vertices (the same Cayley\u0026rsquo;s formula that reappears later, in the spanning tree section), and in every one of them, without exception, the number of edges is exactly the number of vertices minus one.\nNote The converse is also true: a connected graph with \\(n\\) vertices and \\(n-1\\) edges is necessarily a tree. Moreover, any two of the three conditions below imply the third:\n\\(G\\) is connected \\(G\\) is acyclic \\(|E| = |V| - 1\\) Center of a Tree # Before talking about the center of a tree specifically, it is worth recalling the general definition of eccentricity and center of a graph, which we already saw when studying distance, eccentricity, and diameter.\nEccentricity and Center Let \\(G = (V, E)\\) be a graph. The eccentricity of a vertex \\(v\\) is $$e(v) = \\max_{w \\in V} d(v, w)$$ that is, the greatest distance from \\(v\\) to any other vertex of the graph. The center of \\(G\\) is the set $$c(G) = \\{w \\in V \\mid e(w) \\text{ is minimum}\\}$$ In any graph, \\(1 \\leq |c(G)| \\leq |V(G)|\\).\nIn an arbitrary graph, the center can have any size between 1 and all the vertices. In trees, however, this number is always very small:\nTheorem: Center of a Tree The center of any tree consists of a single vertex or of two adjacent vertices.\nProof: the argument rests on three facts.\nFact 1: in a tree with at least three vertices, no leaf belongs to the center. If \\(f\\) is a leaf and \\(u\\) is its only neighbor, every path leaving \\(f\\) goes through \\(u\\); hence \\(d(f, w) = 1 + d(u, w)\\) for every \\(w \\neq f\\), and \\(e(f) = e(u) + 1\\). The leaf is always one unit farther from everything than its neighbor.\nFact 2: if we remove all the leaves of a tree \\(G\\) (and the edges incident to them), obtaining a graph \\(G^{\\prime}\\), every vertex of \\(G^{\\prime}\\) has eccentricity exactly one unit smaller in \\(G^{\\prime}\\) than it had in \\(G\\). This happens because, in a tree, the longest paths from any vertex necessarily end at leaves; removing the leaves shortens these paths by exactly 1.\nFact 3: \\(c(G) = c(G^{\\prime})\\). Since all vertices of \\(G^{\\prime}\\) had their eccentricity reduced by the same amount (Fact 2), and the removed leaves were not in the center (Fact 1), the set of vertices of minimum eccentricity does not change.\nFacts 1, 2, and 3 justify the following iterative algorithm for finding the center:\n1 Remove the leaves Remove all the leaves of \\(G\\) (along with the incident edges). 2 Repeat Repeat the procedure on the remaining graph. 3 Stop at one or two vertices Stop when a single vertex remains, or two vertices joined by an edge: that is the center. Since the graph has a finite number of vertices and each round removes at least one leaf, the process terminates, and it can only stop at a single vertex or at two vertices joined by an edge (if three or more vertices were left, at least two of them would still be leaves, and the next round would remove them). Therefore, the center of any tree is a single vertex or two adjacent vertices, which is what we wanted to prove. \\(\\blacksquare\\)\nBalanced trees # A neighboring concept, widely used in data structures, is that of a balanced tree: a rooted tree in which the subtrees hanging from each vertex have similar sizes, so that no branch is much longer than the others. The exact criterion changes from one structure to another. In AVL trees (named after the initials of Georgy Adelson-Velsky and Evgenii Landis, who published them in 1962), the heights of the two subtrees of any vertex differ by at most 1; in red-black trees, a rule about vertex colors guarantees that no path from the root to a leaf is more than twice as long as another. In both cases, the tree rebalances itself at each insertion or removal, which keeps the height logarithmic in the number of elements, and it is precisely this small height that makes searches, insertions, and removals fast in these structures.\nThe figure below shows three balanced trees:\nBalanced trees: the subtrees of each vertex have similar sizes Why balancing matters: balanced vs. degenerate height \u0026ldquo;Logarithmic height\u0026rdquo; is an abstract phrase until you see how big the difference is in practice. Let us compare the height of a balanced binary tree (generated with all levels complete) with that of a degenerate tree, a chain of only-children, like a linked list, which is what you get, for example, by inserting already sorted data into a plain binary search tree, without rebalancing:\nimport networkx as nx import math for h in [1, 2, 3, 4, 10]: balanced_tree = nx.balanced_tree(r=2, h=h) n = balanced_tree.number_of_nodes() balanced_height = nx.eccentricity(balanced_tree, v=0) # 0 is the root degenerate_height = n - 1 # chain with n vertices, rooted at one end print(f\u0026#34;n={n:\u0026gt;5} | balanced height={balanced_height:\u0026gt;2} | degenerate height={degenerate_height:\u0026gt;5}\u0026#34;) n= 3 | balanced height= 1 | degenerate height= 2 n= 7 | balanced height= 2 | degenerate height= 6 n= 15 | balanced height= 3 | degenerate height= 14 n= 31 | balanced height= 4 | degenerate height= 30 n= 2047 | balanced height=10 | degenerate height= 2046 With only 2,047 elements, the balanced tree has height 10, that is, any search finds what it is looking for in at most 10 steps, while the degenerate version has height 2,046, practically going through all the elements one by one (the same cost as searching a list). It is exactly this difference, which only grows as \\(n\\) increases, that makes balanced trees indispensable in high-performance search structures.\nExamples # Example 1: center of a cross-shaped graph Statement: Find the center of the cross-shaped tree below, with a central vertex \\(c\\) and four \u0026ldquo;arms\u0026rdquo; of lengths 1, 2, 2, and 4 extending from it.\nCenter algorithm on a cross-shaped graph Solution: Applying the algorithm: in each round, we remove the current leaves (the tips of each arm). Since the arms have different lengths, some disappear before others, and the process converges to a single vertex. Note that this vertex is not the central hub \\(c\\) from which the arms extend: it is a vertex in the middle of the longest arm. The center of a tree is defined by minimum eccentricity, not by the \u0026ldquo;visually most central\u0026rdquo; position in the drawing, and that is why the algorithm, and not intuition, is the reliable tool.\nExample 2: center of a branched tree Statement: Find the center of the branched tree below (it is neither a simple path nor a cross).\nCenter algorithm on a branched tree Solution: Applying the same leaf-removal algorithm, the procedure ends with two adjacent vertices. Note that, even in a tree with a much more irregular structure than a path or a cross, the algorithm keeps working in exactly the same way, and the result is still 1 or 2 adjacent vertices, as the theorem guarantees.\nChecking with code Implementing the leaf-removal algorithm by hand is a good exercise, but networkx already ships this computation as nx.center. Let us apply it to the two graphs we have just solved by hand:\nimport networkx as nx G_cross = nx.Graph() G_cross.add_edges_from([ (\u0026#34;c\u0026#34;, \u0026#34;a1\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;b1\u0026#34;), (\u0026#34;b1\u0026#34;, \u0026#34;b2\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e1\u0026#34;), (\u0026#34;e1\u0026#34;, \u0026#34;e2\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d1\u0026#34;), (\u0026#34;d1\u0026#34;, \u0026#34;d2\u0026#34;), (\u0026#34;d2\u0026#34;, \u0026#34;d3\u0026#34;), (\u0026#34;d3\u0026#34;, \u0026#34;d4\u0026#34;), ]) print(\u0026#34;Center of the cross graph:\u0026#34;, sorted(nx.center(G_cross))) G_branched = nx.Graph() G_branched.add_edges_from([ (\u0026#34;c\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;a\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;d\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;f\u0026#34;, \u0026#34;h\u0026#34;), ]) print(\u0026#34;Center of the branched tree:\u0026#34;, sorted(nx.center(G_branched))) Center of the cross graph: [\u0026#39;d1\u0026#39;] Center of the branched tree: [\u0026#39;c\u0026#39;, \u0026#39;d\u0026#39;] Both results match exactly what we found by hand by removing leaves round by round, including the \u0026ldquo;surprise\u0026rdquo; of the first example, in which the center is not the vertex \\(c\\) from which the arms extend, but \\(d_1\\), in the middle of the longest arm.\nSpanning Tree # Spanning Tree A spanning tree of a connected graph \\(G\\) is a spanning subgraph of \\(G\\) that is a tree, that is, a subgraph with all \\(n\\) vertices of \\(G\\) and \\(n-1\\) edges forming a tree.\nEvery spanning tree is obtained from \\(G\\) by removing exactly \\(|E| - (n-1)\\) edges without disconnecting the graph.\nFor every connected graph \\(G\\), it is always possible to find a spanning tree systematically: select edges of \\(G\\), one at a time, so that no cycle is created, and repeat this procedure until \\(n-1\\) edges have been chosen (or, which amounts to the same thing in a connected graph, until no remaining edge can be added without closing a cycle). It is not enough to stop when all vertices appear in some chosen edge: in the path \\(a - b - c - d\\), the edges \\((a,b)\\) and \\((c,d)\\) already touch all four vertices, but they form a forest, not a tree.\nChoosing edges without closing cycles is the most basic principle behind building a spanning tree, and it is from this principle that the best-known algorithms are built:\nPrim\u0026rsquo;s algorithm and Kruskal\u0026rsquo;s algorithm: they apply this same idea of selecting edges without creating cycles, but always choosing the available edge of least weight, to find the minimum spanning tree (the one of least total cost) in weighted graphs BFS (breadth-first search) and DFS (depth-first search): they naturally produce a spanning tree as they traverse the graph, marking as a tree edge each edge used to reach a vertex not yet visited Checking with code Let us see breadth-first search producing a spanning tree in practice, using the same graph \\(G\\) as the examples that follow (vertices \\(a,b,c,d,e,f\\), with \\(a\\) adjacent to \\(b\\), \\(c\\), \\(e\\), and \\(f\\)):\nimport networkx as nx G = nx.Graph() G.add_edges_from([ (\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;d\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;), ]) T_bfs = nx.bfs_tree(G, \u0026#34;a\u0026#34;) print(\u0026#34;Edges of the BFS tree from a:\u0026#34;, sorted(tuple(sorted(e)) for e in T_bfs.edges())) print(\u0026#34;Is it a tree?\u0026#34;, nx.is_tree(T_bfs.to_undirected())) Edges of the BFS tree from a: [(\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;), (\u0026#39;a\u0026#39;, \u0026#39;c\u0026#39;), (\u0026#39;a\u0026#39;, \u0026#39;e\u0026#39;), (\u0026#39;a\u0026#39;, \u0026#39;f\u0026#39;), (\u0026#39;c\u0026#39;, \u0026#39;d\u0026#39;)] Is it a tree? True Breadth-first search from \\(a\\) first visits all the direct neighbors of \\(a\\) (\\(b\\), \\(c\\), \\(e\\), \\(f\\)) and only then reaches \\(d\\), through \\(c\\). The result is a third spanning tree of \\(G\\), different from the two we will see next in Examples 1 and 2, yet another confirmation that the method used to build the spanning tree (manual edge selection, BFS, DFS, \u0026hellip;) determines which specific tree you get.\nExample 1: two different spanning trees Statement: Find two different spanning trees of the graph \\(G\\) below, with vertices \\(a, b, c, d, e, f\\) and containing some cycles.\nT1 and T2 are two different spanning trees of G Solution: \\(T_1\\) and \\(T_2\\) are two distinct spanning trees of \\(G\\), both connected, acyclic, and covering all the vertices of \\(G\\), but using different sets of edges. This illustrates that the spanning tree of a graph is, in general, not unique.\nExample 2: the order in which edges are chosen matters Statement: Apply the edge-selection algorithm to the graph \\(G\\) with the edge list \\(\\{(a,b), (a,c), (b,c), (c,e), (a,e), (e,f), (a,f), (e,d), (c,d)\\}\\), going through it in that order.\nG with named edges Solution: Applying the edge-selection algorithm in this order, avoiding cycles at each step, we obtain the spanning tree \\(T_1\\) from the previous example. If we went through the edge list in another order, we might have chosen different edges at the points where there was more than one cycle-free option, and the result would be a different, though equally valid, spanning tree (this is exactly what happens in Exercise 3 below, which produces \\(T_2\\)).\nHow many spanning trees can a graph have? A connected graph with \\(n\\) vertices can have many distinct spanning trees. For the complete graph \\(K_n\\), Cayley\u0026rsquo;s formula shows that this number is \\(n^{n-2}\\), the same as the number of trees with \\(n\\) labeled vertices, a quantity that grows very rapidly with \\(n\\). The name honors Arthur Cayley, who published a note about it in 1889, but the formula had already been obtained by Carl Wilhelm Borchardt in 1860 (see the Cayley\u0026rsquo;s formula entry). This result goes beyond what we need here, but it helps explain why the choice of edge-selection criterion (such as minimum weight, in the case of Prim and Kruskal) is so important: among exponentially many possible spanning trees, we usually want one specific tree.\nnetworkx computes this number directly (using Kirchhoff\u0026rsquo;s Matrix-Tree Theorem, without having to enumerate the trees one by one), and the result matches Cayley\u0026rsquo;s formula exactly:\nimport networkx as nx for n in [2, 3, 4, 5, 6]: K_n = nx.complete_graph(n) count = nx.number_of_spanning_trees(K_n) formula = n ** (n - 2) print(f\u0026#34;n={n}: nx.number_of_spanning_trees = {count:.0f} | n**(n-2) = {formula}\u0026#34;) n=2: nx.number_of_spanning_trees = 1 | n**(n-2) = 1 n=3: nx.number_of_spanning_trees = 3 | n**(n-2) = 3 n=4: nx.number_of_spanning_trees = 16 | n**(n-2) = 16 n=5: nx.number_of_spanning_trees = 125 | n**(n-2) = 125 n=6: nx.number_of_spanning_trees = 1296 | n**(n-2) = 1296 Rooted Trees # In computing, we often use trees with a designated root. Given a tree \\(T = (V, E)\\), we just pick any vertex \\(v\\) and call it the root. The tree is then called a rooted tree, and its usual graphical representation (root at the top, branches going down) gains a well-defined hierarchy.\nThe graph theory tree and the computing tree For graph theory, a tree is just a connected acyclic graph: it has no root, no order among the neighbors of a vertex, no \u0026ldquo;left\u0026rdquo; and \u0026ldquo;right\u0026rdquo;. The tree in data structures is richer: a binary search tree has a fixed root, the parent/child hierarchy, and the distinction between left child and right child. That is why two different binary trees (the root \\(r\\) with a single child \\(x\\) on the left, and the same root with \\(x\\) on the right) are the same graph: the vertices \\(r\\) and \\(x\\) joined by an edge.\nExample: choosing a root Statement: Consider a tree with no defined root, with vertices \\(a\\) through \\(n\\). Choose vertex \\(d\\) as the root and redraw the tree accordingly.\nChoosing a vertex as the root Solution: by choosing vertex \\(d\\) as the root, the same tree is drawn with \\(d\\) at the top and the other vertices arranged in levels below it.\nWith a root \\(r\\) fixed, we define:\nAncestor, Descendant, Parent, and Child Let \\(v\\) and \\(w\\) be two vertices of a rooted tree \\(T\\) with root \\(r\\). If \\(v\\) belongs to the path from \\(r\\) to \\(w\\), then \\(v\\) is an ancestor of \\(w\\), and \\(w\\) is a descendant of \\(v\\). If, in addition, \\((v, w)\\) is an edge, then \\(v\\) is the parent of \\(w\\) (and \\(w\\) is a child of \\(v\\)).\nThe root is the only vertex with no parent; the leaves are the vertices with no children.\nExample: ancestor, descendant, parent, and child Statement: In the rooted tree \\(T\\) below (root \\(r\\), children \\(b\\) and \\(c\\) of \\(r\\), children \\(d\\) and \\(e\\) of \\(c\\), and children \\(f\\) and \\(g\\) of \\(e\\)), identify the ancestry and kinship relations among some vertices.\nc is an ancestor of d, e, f, g Solution:\n\\(c\\) is an ancestor of \\(d\\), \\(e\\), \\(f\\), and \\(g\\) \\(c\\) is the parent of \\(d\\) and of \\(e\\) \\(b\\) is neither an ancestor nor a descendant of \\(c\\): they are on different branches of the tree \\(b\\) and \\(c\\) are children of the root \\(r\\) The level and the height are also defined from the root:\nLevel and Height Let \\(T\\) be a rooted tree with root \\(r\\). The level of a vertex \\(v\\), denoted \\(\\text{level}(v)\\), is the length of the unique path from \\(r\\) to \\(v\\). The height of \\(T\\) is the greatest level among all vertices: $$\\text{height}(T) = \\max_{v \\in V} \\text{level}(v)$$ Example: level and height Statement: In the same tree as the previous example (root \\(r\\), children \\(b\\) and \\(c\\), grandchildren \\(d\\) and \\(e\\), great-grandchildren \\(f\\) and \\(g\\)), compute the level of each vertex and the height of the tree.\nLevels 0 to 3: height(T) = 3 Solution:\n\\(\\text{level}(r) = 0\\) \\(\\text{level}(b) = \\text{level}(c) = 1\\) \\(\\text{level}(d) = \\text{level}(e) = 2\\) \\(\\text{level}(f) = \\text{level}(g) = 3\\) \\(\\text{height}(T) = 3\\) Recursion: computing the height programmatically The definition of height (\u0026ldquo;the greatest level among all vertices\u0026rdquo;) is natural to compute top-down (from the root to the leaves), but the most direct way to program it is precisely the opposite: a recursive bottom-up function, which asks for the height of each child before answering for itself.\nRepresenting the tree as a dictionary of children (the simplest way to store a rooted tree in Python), the height of a vertex is 0 if it has no children (it is a leaf), or 1 plus the greatest height among its children, otherwise:\nchildren = { \u0026#34;r\u0026#34;: [\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;], \u0026#34;b\u0026#34;: [], \u0026#34;c\u0026#34;: [\u0026#34;d\u0026#34;, \u0026#34;e\u0026#34;], \u0026#34;d\u0026#34;: [], \u0026#34;e\u0026#34;: [\u0026#34;f\u0026#34;, \u0026#34;g\u0026#34;], \u0026#34;f\u0026#34;: [], \u0026#34;g\u0026#34;: [], } def height(node): if not children[node]: return 0 return 1 + max(height(child) for child in children[node]) for node in children: print(f\u0026#34;height({node}) = {height(node)}\u0026#34;) height(r) = 3 height(b) = 0 height(c) = 2 height(d) = 0 height(e) = 1 height(f) = 0 height(g) = 0 Note that height(r) = 3 is exactly the height of the entire tree \\(T\\) (the root recursively \u0026ldquo;asks\u0026rdquo; for the height of the whole structure below it), and that height(b) = height(d) = height(f) = height(g) = 0 for all the leaves, which matches the fact that the level of a leaf is the greatest \u0026ldquo;local\u0026rdquo; level of its own subtree (which is just itself). This is the same recursive idea behind almost every algorithm that processes trees: solve the problem on the subtrees first, and then combine the answers at the root.\nThese concepts appear everywhere in computer science: whenever a structure organizes data hierarchically, there is a rooted tree under the hood. The most literal example of all is precisely the one that opened this article: a file system.\nApplication: the tree behind a file system Every file system is a rooted tree: the root is the main folder, each folder is the child of exactly one parent folder, and files are the leaves. Let us build a small real project structure (in a temporary folder, so the code runs on any machine) and compute its levels and its height with networkx:\nimport os import tempfile from pathlib import Path import networkx as nx with tempfile.TemporaryDirectory() as tmp: root = Path(tmp) / \u0026#34;project\u0026#34; (root / \u0026#34;src\u0026#34;).mkdir(parents=True) (root / \u0026#34;tests\u0026#34;).mkdir() (root / \u0026#34;src\u0026#34; / \u0026#34;main.py\u0026#34;).touch() (root / \u0026#34;src\u0026#34; / \u0026#34;utils.py\u0026#34;).touch() (root / \u0026#34;tests\u0026#34; / \u0026#34;test_main.py\u0026#34;).touch() (root / \u0026#34;README.md\u0026#34;).touch() G = nx.Graph() for current_folder, subdirs, files in os.walk(root): current_folder = Path(current_folder) for name in subdirs + files: source = str(current_folder.relative_to(root.parent)) target = str((current_folder / name).relative_to(root.parent)) G.add_edge(source, target) print(\u0026#34;Is it a tree?\u0026#34;, nx.is_tree(G)) levels = nx.shortest_path_length(G, root.name) for node, level in sorted(levels.items(), key=lambda x: (x[1], x[0])): print(f\u0026#34;level({node}) = {level}\u0026#34;) print(\u0026#34;height(project) =\u0026#34;, max(levels.values())) Is it a tree? True level(project) = 0 level(project/README.md) = 1 level(project/src) = 1 level(project/tests) = 1 level(project/src/main.py) = 2 level(project/src/utils.py) = 2 level(project/tests/test_main.py) = 2 height(project) = 2 os.walk traverses the directory tree from top to bottom, exactly as we did by hand in the previous examples, only now on a real file structure, with pathlib taking care of the paths. Each folder or file nested one level deeper increases the \u0026ldquo;level\u0026rdquo; by 1, and the height of the tree is, as always, the greatest of these values.\nRooted trees in computing Binary search tree: each node has at most 2 children; node in the left subtree \u0026lt; node \u0026lt; node in the right subtree Heap: nearly complete binary tree; parent node ≥ children (max-heap) or parent node ≤ children (min-heap) Trie: tree for storing strings, with paths from the root to each node representing prefixes Decision tree: used in machine learning; each internal node represents a decision condition, and the level at which it appears reflects its order of evaluation Exercises # The exercises below are in increasing order of difficulty: the first two reinforce the basic properties of trees and forests, and the third revisits the spanning tree from the previous example.\nExercise 1: Number of edges Statement: How many edges does a forest with 10 vertices and 3 connected components have?\nSolution Each connected component with \\(n_i\\) vertices is a tree with \\(n_i - 1\\) edges. If the components have \\(n_1, n_2, n_3\\) vertices with \\(n_1 + n_2 + n_3 = 10\\), the total number of edges is \\((n_1 - 1) + (n_2 - 1) + (n_3 - 1) = 10 - 3 = 7\\) edges.\nExercise 2: Uniqueness of the path Statement: In a tree with 6 vertices forming the path \\(1 - 2 - 3 - 4 - 5 - 6\\) with an extra edge \\(\\{2, 5\\}\\), is the resulting graph still a tree?\nSolution No. The graph has 6 vertices and 6 edges (the path has 5 + the extra edge = 6). A tree with 6 vertices has exactly 5 edges. Besides, there is a cycle: \\(2 - 3 - 4 - 5 - 2\\). Therefore, it is not a tree.\nExercise 3: another order, another spanning tree Statement: Revisit the graph \\(G\\) with the edge list \\(\\{(a,b), (a,c), (b,c), (c,e), (a,e), (e,f), (a,f), (e,d), (c,d)\\}\\) from Example 2 of the spanning tree section. Choose a different order for this edge list and find a spanning tree of \\(G\\) different from the one obtained earlier.\nSolution For example, going through the edges in the order \\((a,e), (e,f), (e,d), (a,b), (a,c), (c,e), (b,c), (a,f), (c,d)\\), accepting each edge that does not close a cycle with those already chosen, we obtain the edge set \\(\\{(a,e), (e,f), (e,d), (a,b), (a,c)\\}\\), which already connects the 6 vertices with exactly 5 edges: a spanning tree different from that of Example 2. Any other order that yields a tree with the same properties (connected, acyclic, 5 edges, 6 vertices) is also correct; the important thing is to realize that the order in which edges are chosen determines which specific spanning tree you get.\nSummary # Concept Description Tree Connected and acyclic graph Forest Acyclic graph (each component is a tree) Leaf Vertex of degree 1 Theorem 1 There is a unique path between each pair of vertices \\(\\Leftrightarrow\\) the graph is a tree Theorem 2 (\\(m = n-1\\)) Every tree with \\(n\\) vertices has \\(n-1\\) edges Eccentricity / Center \\(e(v) = \\max_w d(v,w)\\); \\(c(G)\\) = vertices of minimum eccentricity Center of a tree 1 vertex, or 2 adjacent vertices; found by iteratively removing leaves Balanced tree Subtrees of each vertex with similar sizes (in AVL trees, heights that differ by at most 1); logarithmic height Spanning tree Spanning subgraph that is a tree; in general not unique Root Special vertex in rooted trees Ancestor / Descendant / Parent / Child Relations defined by the unique path from the root to each vertex Level / Height \\(\\text{level}(v)\\) = length of the path from the root to \\(v\\); \\(\\text{height}(T) = \\max_v \\text{level}(v)\\) Next Steps # Trees are the minimal skeleton that keeps a graph connected, but, because they are acyclic, they cannot answer questions about graphs that do have cycles and where we want to traverse the whole structure in some specific way: visiting each edge exactly once, or visiting each vertex exactly once. These questions motivate Eulerian and Hamiltonian graphs, the topic of the next article in the series: Eulerian trails (which traverse all the edges of a graph) and Hamiltonian cycles (which visit all the vertices), including the traveling salesman problem.\n","date":"2026-08-18","externalUrl":null,"permalink":"/2026/08/trees/","section":"Blog","summary":"A file system is a tree: a connected graph with no cycles. We prove that every tree has one edge fewer than it has vertices and that its center has one or two vertices, and we look at spanning trees and rooted trees, including the difference between a balanced tree and a degenerate one.","title":"Trees: The Most Elegant Structure in Graphs","type":"posts"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/caminhos/","section":"tags","summary":"","title":"caminhos","type":"tags"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/ciclos/","section":"tags","summary":"","title":"ciclos","type":"tags"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/conexidade/","section":"tags","summary":"","title":"conexidade","type":"tags"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/connectivity/","section":"tags","summary":"","title":"connectivity","type":"tags"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/cycles/","section":"tags","summary":"","title":"cycles","type":"tags"},{"content":"","date":"2026-08-13","externalUrl":null,"permalink":"/tags/paths/","section":"tags","summary":"","title":"paths","type":"tags"},{"content":" How do we know whether two points of a network are connected, and by which shortest path? So far we have studied graphs as static objects, sets of vertices and edges. But the most common question about a graph is how to navigate it: starting from a vertex and following edges to reach others.\nThis leads us to the concepts of walks, trails, paths, and cycles, and to the notion of connectivity.\nWhy Do Paths and Cycles Matter? # Navigating a graph, and knowing whether that navigation is even possible, is the basis of central problems in computer science and applied mathematics:\nNetwork routing: routing protocols (such as those used on the internet) need to find paths between routers, and the concept of shortest distance directly defines the cost of a route. Distributed systems: before running any distributed algorithm, it is common to check whether the network of machines is connected; if it is not, the system splits into partitions that cannot communicate. Resource allocation: matching problems (such as assigning workers to machines, or candidates to openings) are modeled as bipartite graphs, and solving them depends on understanding the structure of these graphs. Verification of circuits and protocols: detecting cycles in a dependency graph is what prevents, for example, deadlocks in concurrent systems or circular references in software builds. Walks, Trails, and Paths # There are different ways to traverse a graph, and the distinctions between them matter:\nWalk A walk in \\(G\\) is a sequence of vertices $$P = v_0, v_1, \\ldots, v_k$$ such that \\((v_i, v_{i+1}) \\in E(G)\\) for \\(0 \\leq i \\leq k-1\\), that is, between each pair of consecutive vertices of the sequence there is an edge of \\(G\\).\nThe integer \\(k\\) (the number of edges traversed) is the length of the walk, \\(v_0\\) is its start, \\(v_k\\) is its end, and we say that \\(P\\) is a walk from \\(v_0\\) to \\(v_k\\).\nIn a walk, vertices and edges may repeat.\nTrail A trail is a walk in which no edge repeats (but vertices may repeat).\nPath A path is a walk in which no vertex repeats (and therefore no edge repeats either).\nThe hierarchy is: every path is a trail, and every trail is a walk. The converse is not necessarily true.\nTo fix these ideas, we will use the same graph \\(G\\) in several examples throughout this article, to observe how the same eight vertices support increasingly specific concepts, from walk to graph center. We start from the graph \\(G\\) below:\nGraph G: reference for the examples in this section Example: walk, trail, and path in the same graph Statement: In the graph \\(G\\) above, classify the sequences \\(P_1 = a, b, c, d, c, b, g\\), \\(P_2 = a, b, c, f, b, g\\), \\(P_3 = a, h, g\\), and \\(P_4 = g, h, a, b, f, e, c, d\\) as walk, trail, or path.\nSolution:\n\\(P_1\\) is a walk from \\(a\\) to \\(g\\): the vertices \\(b\\) and \\(c\\) and the edges \\(\\{b,c\\}\\) and \\(\\{c,d\\}\\) appear twice. \\(P_2\\) is a trail from \\(a\\) to \\(g\\): no edge repeats, but the vertex \\(b\\) appears twice, so \\(P_2\\) is not a path. \\(P_3\\) is a path between \\(a\\) and \\(g\\): no vertex repeats. \\(P_4\\) is also a path, now between \\(g\\) and \\(d\\), passing through all the other vertices of \\(G\\) without repeating any. The figure below highlights \\(P_1\\) over the graph \\(G\\), showing where the vertex \\(c\\) and the edge \\(\\{c,d\\}\\) repeat:\nP1 = a,b,c,d,c,b,g: a walk that repeats a vertex and an edge Closed Walks and Closed Trails # A walk (or trail, or path) is closed if the initial vertex coincides with the final vertex, that is, \\(v_0 = v_k\\).\nClosed Trail A closed trail is a closed walk in which all edges are distinct.\nCycle A cycle is a closed walk in which, in addition, all vertices are distinct, except for the first and the last, which coincide by definition. In other words, if \\(v_0 = v_k\\) and \\(v_0, v_1, \\ldots, v_{k-1}\\) is a path, then \\(P\\) is a cycle.\nThe length of a cycle is given by its number of edges (equal to its number of distinct vertices).\nEven Cycle and Odd Cycle A cycle with an odd number of edges is called an odd cycle; otherwise, it is called an even cycle.\nCycles of length 3 are triangles.\nThe graph \\(G\\) has examples of both kinds: a cycle of length 4 (\\(a,b,g,h,a\\)) and a triangle, which is a cycle of length 3 (\\(a,b,h,a\\)):\nEven cycle a,b,g,h,a vs. odd cycle a,b,h,a Example: closed walk, closed trail, and cycles Statement: Still in the graph \\(G\\), classify \\(P_5 = a, b, c, d, c, e, f, b, g, h, a\\), \\(P_6 = a, b, c, e, f, b, g, h, a\\), \\(P_7 = a, b, g, h, a\\), and \\(P_8 = a, b, h, a\\).\nSolution:\n\\(P_5\\) is a closed walk, but not a closed trail: the edge \\(\\{c,d\\}\\) is traversed twice. \\(P_6\\) is a closed trail (no edge repeats), but it is not a cycle, since the vertex \\(b\\) is visited twice. \\(P_7\\) is an even cycle, of length 4. \\(P_8\\) is an odd cycle, of length 3 (a triangle). Application: detecting circular dependencies Cycles show up (and cause headaches) whenever a software system models dependencies between components as a directed graph: if module \\(A\\) depends on \\(B\\), we draw an edge \\(A \\to B\\). A project only compiles (or imports, or installs) if this dependency graph has no cycles; otherwise, \u0026ldquo;what to load first\u0026rdquo; becomes a question with no answer.\nimport networkx as nx deps = nx.DiGraph() deps.add_edges_from([ (\u0026#34;app\u0026#34;, \u0026#34;auth\u0026#34;), (\u0026#34;app\u0026#34;, \u0026#34;database\u0026#34;), (\u0026#34;auth\u0026#34;, \u0026#34;database\u0026#34;), (\u0026#34;database\u0026#34;, \u0026#34;config\u0026#34;), (\u0026#34;config\u0026#34;, \u0026#34;utils\u0026#34;), (\u0026#34;utils\u0026#34;, \u0026#34;auth\u0026#34;), # closes the cycle auth -\u0026gt; database -\u0026gt; config -\u0026gt; utils -\u0026gt; auth ]) try: cycle = nx.find_cycle(deps) print(\u0026#34;Circular dependency found:\u0026#34;) for source, target in cycle: print(f\u0026#34; {source} -\u0026gt; {target}\u0026#34;) except nx.NetworkXNoCycle: print(\u0026#34;No cycle: the dependency graph is a DAG, safe build.\u0026#34;) Circular dependency found: auth -\u0026gt; database database -\u0026gt; config config -\u0026gt; utils utils -\u0026gt; auth The graph below shows the declared dependencies, with the cycle found highlighted in orange:\nDependency graph with the cycle detected by nx.find_cycle The cycle is not obvious when looking module by module: auth imports database, which imports config, which imports utils, which in turn imports auth back. This is exactly the kind of indirect circular dependency that build systems and package managers such as pip and npm traverse as a graph, instead of relying on someone to notice the problem manually.\nWhat each tool does when it finds a cycle varies: cargo (Rust) refuses the build outright (it only accepts cycles between dev-dependencies), like many compiled build systems, since compiling requires a strict order between dependencies; classic make detects it, warns, and breaks the cycle arbitrarily to move on; and interpreted languages such as Python and Node.js tolerate the circular import at run time, handing over a partially initialized module at the point where the cycle closes, a classic source of subtle bugs. Behind all these strategies lies the same fact from graph theory: a valid build, install, or import order is a topological sort, and a topological sort only exists if the directed graph is acyclic, which is why these tools need to check whether sorting is even possible, even when they decide to tolerate the cycle.\nConnectivity # So far we have said \u0026ldquo;there is a path between \\(v\\) and \\(w\\)\u0026rdquo; loosely. Let us formalize this idea pair by pair before applying it to the whole graph.\nConnected Vertices Two vertices \\(v\\) and \\(w\\) are connected in \\(G\\) when there is some path between \\(v\\) and \\(w\\) in \\(G\\).\nNow consider the graph \\(H\\) below, formed by a cluster of six vertices \\(\\{p,q,r,s,t,u\\}\\) and, separately, a triangle \\(\\{x,y,z\\}\\):\nGraph H: two clusters with no edges between them Example: checking connectivity pair by pair Statement: In the graph \\(H\\), are \\(p\\) and \\(t\\) connected? And \\(p\\) and \\(x\\)?\nSolution:\n\\(p\\) and \\(t\\) are connected: the path \\(P = p, r, s, t\\) links the two. \\(p\\) and \\(x\\) are not connected: there is no path between them, since no edge links the cluster \\(\\{p,\\ldots,u\\}\\) to the triangle \\(\\{x,y,z\\}\\). Connected Graph A graph \\(G\\) is connected when every pair of distinct vertices of \\(G\\) is connected, that is, when there is a path between any pair of vertices \\(u, v \\in V(G)\\). Otherwise, \\(G\\) is disconnected.\nThe graph \\(H\\) of the example above is disconnected, but each of its two parts, on its own, is connected. This leads us to the definition of connected component.\nConnected Component Let \\(G = (V, E)\\) be a graph and \\((V_1, V_2, \\ldots, V_k)\\) a partition of \\(V\\) (that is, \\(V_1 \\cup V_2 \\cup \\cdots \\cup V_k = V\\) and \\(V_i \\cap V_j = \\emptyset\\) for \\(i \\neq j\\)). The induced subgraph \\(G[V_i]\\) is a connected component of \\(G\\) if:\n\\(G[V_i]\\) is connected, and for every vertex \\(v \\in V - V_i\\), the subgraph \\(G[V_i \\cup \\{v\\}]\\) is not connected. Condition 2 is what forces each component to be the largest possible connected subgraph: if any outside vertex is added to \\(V_i\\), the result stops being connected. It is in this sense that connected components are the \u0026ldquo;maximal connected pieces\u0026rdquo; of \\(G\\).\nWe denote by \\(\\omega(G)\\) the number of connected components of \\(G\\). A graph is connected if and only if \\(\\omega(G) = 1\\).\nWatch out for the notation \\(\\omega(G)\\) for the number of components is common in Portuguese-language texts, but it is not universal: many English books write \\(c(G)\\) or \\(k(G)\\) and reserve \\(\\omega(G)\\) for the clique number, the size of the largest clique of \\(G\\). It is worth checking the convention when consulting another source. In this article, \\(c(G)\\) is the center of the graph, defined later.\nWhy this matters Many graph algorithms assume, as a precondition, that the input graph is connected. When that is not the case, a common step is to \u0026ldquo;preprocess\u0026rdquo; the graph to split it into its connected components and solve the problem in each one of them separately.\nBack to the graph \\(H\\), we highlight its two connected components below:\nThe two connected components of H Example: counting connected components Statement: How many connected components does the graph \\(H\\) have? And a graph \\(G^{\\prime}\\) with \\(V(G^{\\prime}) = \\{1, \\ldots, 10\\}\\) partitioned into \\(V_1 = \\{1,2,3,4\\}\\) (with edges \\(\\{1,2\\},\\{1,4\\},\\{2,3\\},\\{2,4\\},\\{3,4\\}\\)), \\(V_2 = \\{5,6\\}\\) (with the edge \\(\\{5,6\\}\\)), \\(V_3 = \\{7\\}\\) (isolated vertex), and \\(V_4 = \\{8,9,10\\}\\) (a triangle)?\nSolution: In the graph \\(H\\), the subgraph induced by \\(\\{p,q,r,s,t,u\\}\\) is connected, and the subgraph induced by \\(\\{x,y,z\\}\\) is also connected, and no vertex can be moved from one to the other without breaking connectivity. Hence these are the two connected components of \\(H\\), and \\(\\omega(H) = 2\\).\nA graph can have many components, including isolated vertices. Each \\(G^{\\prime}[V_i]\\) is connected and maximal, so \\(\\omega(G^{\\prime}) = 4\\).\nThe figure below shows \\(G^{\\prime}\\), with each connected component in a different color:\nA graph with four connected components, including an isolated vertex And what about the graph \\(G\\) we saw in the previous section (vertices \\(\\{a,\\ldots,h\\}\\))? Every pair of vertices of \\(G\\) is connected. For example, \\(P_4\\) already showed a path between \\(g\\) and \\(d\\) passing through all the other vertices. Hence \\(G\\) is connected and \\(\\omega(G) = 1\\).\nDistance, Eccentricity, and Diameter # With the concept of connectivity in hand, we can measure \u0026ldquo;distances\u0026rdquo; within a graph:\nDistance The distance \\(d(v, w)\\) between two vertices \\(v\\) and \\(w\\) is the length of the shortest path between them, which is also called a geodesic. If \\(v\\) and \\(w\\) are not connected (that is, they are in distinct connected components of \\(G\\)), there is no path between them, and we define \\(d(v,w) = \\infty\\).\nThe following definitions (eccentricity, diameter, and center) assume that \\(G\\) is a connected graph (otherwise, some \\(e(v)\\) could be infinite).\nEccentricity Let \\(G\\) be a connected graph. The eccentricity \\(e(v)\\) of a vertex \\(v\\) is the greatest distance from \\(v\\) to any other vertex: $$e(v) = \\max_{w \\in V} d(v, w)$$ Diameter The diameter of \\(G\\) is the greatest eccentricity: $$\\text{diam}(G) = \\max_{v \\in V} e(v) = \\max_{v,w \\in V} d(v,w)$$ Center The center of \\(G\\) is the set of vertices with the smallest eccentricity: $$c(G) = \\{v \\in V : e(v) = \\min_{u \\in V} e(u)\\}$$Let us compute these three measures in the graph \\(G\\) (vertices \\(\\{a,\\ldots,h\\}\\)) that we used in the previous sections.\nExample: eccentricity, diameter, and center of \\(G\\) Statement: Compute the eccentricity of each vertex of \\(G\\), the diameter of \\(G\\), and its center.\nSolution: Starting from \\(a\\), the distances to the other vertices are: \\(d(a,b)=1\\), \\(d(a,h)=1\\), \\(d(a,g)=2\\), \\(d(a,c)=2\\), \\(d(a,f)=2\\), \\(d(a,e)=3\\), \\(d(a,d)=3\\). The greatest of them is 3, so \\(e(a) = 3\\).\nRepeating the computation for all vertices of \\(G\\), we obtain:\n$$e(a) = e(d) = e(e) = e(g) = e(h) = 3 \\qquad e(b) = e(c) = e(f) = 2$$Since the greatest eccentricity is 3, we have \\(\\text{diam}(G) = 3\\). And since \\(b\\), \\(c\\), and \\(f\\) are the vertices of smallest eccentricity (2), the center is \\(c(G) = \\{b, c, f\\}\\), exactly the three vertices that sit between the \u0026ldquo;left side\u0026rdquo; (\\(a,g,h\\)) and the \u0026ldquo;right side\u0026rdquo; (\\(d,e\\)) of \\(G\\), which makes intuitive sense: vertices through which the paths between the two parts of the graph pass tend to be closer to everyone.\nThe following figure shows the eccentricity of each vertex and highlights the center:\nEccentricities of G and its center {b,c,f} Example: the case of the path \\(P_5\\) Statement: Compute the eccentricity of each vertex, the diameter, and the center of the path \\(P_5 = v_1 - v_2 - v_3 - v_4 - v_5\\).\nSolution: In a path graph, the eccentricity grows quite predictably from the ends:\n\\(e(v_1) = e(v_5) = 4\\) (the ends must cross the whole path) \\(e(v_2) = e(v_4) = 3\\) \\(e(v_3) = 2\\) (the central vertex) Diameter = 4. Center = \\(\\{v_3\\}\\). In any \\(P_n\\), the center is always the vertex (or vertices) closest to the middle of the sequence.\nChecking with code Everything we computed by hand for the graph \\(G\\) (eccentricities, diameter, and center) comes out in a few lines with networkx:\nimport networkx as nx G = nx.Graph() G.add_edges_from([ (\u0026#34;a\u0026#34;,\u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;,\u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;,\u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;,\u0026#34;g\u0026#34;), (\u0026#34;g\u0026#34;,\u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;,\u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;,\u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;,\u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;,\u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;,\u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;,\u0026#34;d\u0026#34;), ]) print(\u0026#34;Eccentricities:\u0026#34;, nx.eccentricity(G)) print(\u0026#34;Diameter:\u0026#34;, nx.diameter(G)) print(\u0026#34;Center:\u0026#34;, nx.center(G)) Eccentricities: {\u0026#39;a\u0026#39;: 3, \u0026#39;b\u0026#39;: 2, \u0026#39;h\u0026#39;: 3, \u0026#39;g\u0026#39;: 3, \u0026#39;c\u0026#39;: 2, \u0026#39;f\u0026#39;: 2, \u0026#39;e\u0026#39;: 3, \u0026#39;d\u0026#39;: 3} Diameter: 3 Center: [\u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;f\u0026#39;] The values match the manual computation exactly. Besides avoiding arithmetic mistakes, the same code works with no change for a graph with 8 or with 8 million vertices; what we did by hand for \\(G\\) is only feasible by hand because the graph is small.\nComputing Distances: Breadth-First Search # In the examples above, we read the distances off the drawing of \\(G\\), and networkx did the same computation in one line. But how do we compute the distance between two vertices of a graph with millions of them? When the edges have no weights, the answer is breadth-first search (BFS).\nThe idea is to explore the graph in layers from a source \\(v\\): first \\(v\\) itself, at distance 0; then all its neighbors, at distance 1; then the neighbors of those neighbors that have not appeared yet, at distance 2; and so on. A queue guarantees this order: each discovered vertex enters at the rear of the queue, and vertices are processed in the order in which they entered.\nBreadth-First Search (BFS) Given a source vertex \\(v\\):\nMark \\(d(v, v) = 0\\) and put \\(v\\) in the queue. While the queue is not empty, remove the first vertex \\(u\\) from the queue. For each neighbor \\(w\\) of \\(u\\) that does not yet have a distance marked, mark \\(d(v, w) = d(v, u) + 1\\) and put \\(w\\) at the rear of the queue. At the end, each vertex reachable from \\(v\\) has its distance marked; those left unmarked are in another connected component, at distance \\(\\infty\\).\nWhy is the marked distance the smallest possible? Vertices leave the queue in increasing order of distance (all those in layer \\(k\\) before any of layer \\(k+1\\)), so the first time a vertex \\(w\\) appears is from a vertex in the layer closest possible to the source. And the cost is low: each vertex enters the queue only once, and each edge is examined twice, once from each endpoint, which gives time \\(O(n + m)\\).\nApplied to the graph \\(G\\) from \\(a\\), BFS reproduces the distances we computed by hand:\nfrom collections import deque import networkx as nx G = nx.Graph() G.add_edges_from([ (\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;), (\u0026#34;a\u0026#34;, \u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;g\u0026#34;), (\u0026#34;g\u0026#34;, \u0026#34;h\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (\u0026#34;b\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;e\u0026#34;), (\u0026#34;e\u0026#34;, \u0026#34;f\u0026#34;), (\u0026#34;c\u0026#34;, \u0026#34;d\u0026#34;), ]) def distances(G, source): \u0026#34;\u0026#34;\u0026#34;Distance from source to each reachable vertex, by breadth-first search.\u0026#34;\u0026#34;\u0026#34; dist = {source: 0} queue = deque([source]) while queue: v = queue.popleft() for w in G.neighbors(v): if w not in dist: # first time w appears dist[w] = dist[v] + 1 queue.append(w) return dist d_a = distances(G, \u0026#34;a\u0026#34;) print(d_a) # Output: {\u0026#39;a\u0026#39;: 0, \u0026#39;b\u0026#39;: 1, \u0026#39;h\u0026#39;: 1, \u0026#39;g\u0026#39;: 2, \u0026#39;c\u0026#39;: 2, \u0026#39;f\u0026#39;: 2, \u0026#39;e\u0026#39;: 3, \u0026#39;d\u0026#39;: 3} print(max(d_a.values())) # 3, the eccentricity of a The figure below draws \\(G\\) in these layers. The thick edges are the ones the search used to discover each vertex; the dashed ones were examined and discarded, because the vertex on the other side already had a distance marked:\nBreadth-first search from a: layers by distance and the search tree The greatest distance found is the eccentricity of \\(a\\). Running BFS from each vertex, we obtain all the eccentricities (and, with them, the diameter and the center) in time \\(O(n(n + m))\\). The same search comes back later in this article, to test whether a graph is bipartite, and in the article on trees, where it produces a spanning tree. It was described by Konrad Zuse in 1945, in a thesis that would only be published in 1972, and reinvented by Edward F. Moore in 1959 to find the way out of mazes (see the entry Breadth-first search).\nWeighted graphs: Dijkstra\u0026rsquo;s algorithm On a street map, edges have different lengths, and the path with the fewest edges is not always the shortest. In a weighted graph, each edge has a weight (distance, time, cost), and the length of a path becomes the sum of the weights of its edges. BFS, which only counts edges, no longer works.\nDijkstra\u0026rsquo;s algorithm, conceived by Edsger W. Dijkstra in 1956 and published in 1959, adapts the same idea: it swaps the ordinary queue for a priority queue and, at each step, fixes the distance of the not-yet-fixed vertex that is closest to the source. The algorithm requires nonnegative weights and, with the priority queue built on a heap, runs in time \\(O((n + m) \\log n)\\). Routing apps use variations of it, with many optimizations for country-sized graphs.\nIn the graph below, the path with the fewest edges from \\(A\\) to \\(B\\) is the direct edge, of weight 4; the shortest one goes through \\(C\\) and \\(D\\) and has length 3:\nimport networkx as nx G = nx.Graph() G.add_weighted_edges_from([(\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, 4), (\u0026#34;A\u0026#34;, \u0026#34;C\u0026#34;, 1), (\u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, 1), (\u0026#34;D\u0026#34;, \u0026#34;B\u0026#34;, 1)]) print(nx.shortest_path(G, \u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;)) # [\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;] (unweighted: BFS) print(nx.dijkstra_path(G, \u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;)) # [\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;, \u0026#39;D\u0026#39;, \u0026#39;B\u0026#39;] print(nx.dijkstra_path_length(G, \u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;)) # 3 Special Graphs # Some graphs appear so frequently in applications and examples that it is worth naming them and recognizing them at once.\nComplete Graph \\(K_n\\) and Trivial Graph The complete graph \\(K_n\\) has \\(n\\) vertices, with every pair of vertices joined by an edge. In particular, \\(K_1\\), the complete graph on a single vertex, with no edges, is called the trivial graph.\nNull Graph \\(N_n\\) The null graph \\(N_n\\) has \\(n\\) vertices and no edges, the opposite extreme of the complete graph.\nThe two extreme cases, side by side:\nTrivial graph K1 and null graph N4 Cycles and Paths # Cycle \\(C_n\\) and Path \\(P_n\\) \\(C_n\\): cycle with \\(n\\) vertices (\\(n \\geq 3\\)), 2-regular \\(P_n\\): path with \\(n\\) vertices (\\(n-1\\) edges), with two vertices of degree 1 and \\(n-2\\) of degree 2 The first cycles of the family \\(C_n\\):\nCycle graphs C3 to C6 And the first paths of the family \\(P_n\\):\nPath graphs P2 to P5 Bipartite Graphs # An important class of graphs is the one in which the vertices can be divided into two groups, with edges only between the groups.\nBipartite Graph A graph \\(G = (V, E)\\) is bipartite if \\(V\\) can be partitioned into two subsets \\(V_1\\) and \\(V_2\\) (with \\(V_1 \\cup V_2 = V\\) and \\(V_1 \\cap V_2 = \\emptyset\\)) such that every edge of \\(G\\) has one endpoint in \\(V_1\\) and the other in \\(V_2\\). We say that \\((V_1, V_2)\\) is a bipartition of \\(V\\).\n\\(V_1\\) and \\(V_2\\) are independent Since every edge links \\(V_1\\) to \\(V_2\\), there can be no edge between two vertices of \\(V_1\\), nor between two vertices of \\(V_2\\). That is, \\(V_1\\) and \\(V_2\\) are independent sets in \\(G\\). This observation will be useful to understand why cycles in bipartite graphs are always even, later on.\nA useful way to visualize a bipartite graph is to draw \\(V_1\\) and \\(V_2\\) as two separate columns, making explicit that every edge crosses from one column to the other:\nGeneric bipartition: edges only between V1 and V2 Example: assigning workers to machines Statement: How do we model, as a bipartite graph, the problem of knowing which workers in a factory know how to operate which machines?\nSolution: Let the workers \\(x_1, x_2, \\ldots, x_r\\) form \\(V_1\\) and the machines \\(y_1, y_2, \\ldots, y_s\\) form \\(V_2\\). There is an edge between worker \\(x_i\\) and machine \\(y_j\\) whenever \\(x_i\\) is able to operate \\(y_j\\). This graph is bipartite by construction: there is never an edge between two workers or between two machines, only between a worker and a machine. This is the kind of structure that matching problems exploit to decide, for example, who should operate what.\nBipartite assignment graph: workers x machines Complete Bipartite Graph \\(K_{p,q}\\) The complete bipartite graph \\(K_{p,q}\\) has \\(|V_1| = p\\), \\(|V_2| = q\\), and every possible edge between \\(V_1\\) and \\(V_2\\). Therefore, \\(|E(K_{p,q})| = p \\cdot q\\).\nA small example is \\(K_{2,3}\\), with all the \\(2 \\times 3 = 6\\) possible edges between the two sides:\nK2,3: complete bipartite graph Examples of bipartite graphs:\nPaths \\(P_n\\) (for \\(n \\geq 2\\)): even-position vertices in \\(V_1\\), odd-position ones in \\(V_2\\) Even cycles \\(C_{2k}\\): vertices alternating between \\(V_1\\) and \\(V_2\\) The following figure shows \\(C_4\\) drawn in two ways (as a polygon and as a bipartition) and brings \\(C_5\\) as a challenge: try to find a valid bipartition for it (spoiler: there is none, as the next theorem will explain).\nC4 in the two equivalent forms, and C5 as a challenge A classic case: \\(K_{3,3}\\) The complete bipartite graph \\(K_{3,3}\\) is the graph of the \u0026ldquo;three houses and three utilities problem\u0026rdquo;: three houses that must be connected to water, electricity, and gas without any line crossing. It is a classic example used to discuss graph planarity, the topic of an upcoming article in this series.\nCharacterization of Bipartite Graphs # Deciding whether a graph is bipartite or not is an important problem. The following theorem shows that it suffices to look for odd cycles.\nTheorem A graph \\(G\\) is bipartite if and only if it contains no cycle of odd length.\nProof (\\(\\Rightarrow\\)): Let \\(G\\) be a bipartite graph with bipartition \\((V_1, V_2)\\), and let \\(C = v_1, v_2, \\ldots, v_k, v_1\\) be a cycle in \\(G\\). Suppose \\(v_1 \\in V_1\\). Since \\(V_1\\) and \\(V_2\\) are independent, each edge of the cycle switches sides: \\(v_2 \\in V_2\\), \\(v_3 \\in V_1\\), \\(v_4 \\in V_2\\), and so on; in general, \\(v_{2i-1} \\in V_1\\) and \\(v_{2i} \\in V_2\\). Since \\((v_k, v_1) \\in E\\) and \\(v_1 \\in V_1\\), the vertex \\(v_k\\) must be in \\(V_2\\) (because \\(G\\) is bipartite, and an edge never links two vertices on the same side). Hence \\(k = 2i\\) for some \\(i\\), that is, \\(k\\) is even and therefore \\(C\\) is an even cycle. Since this holds for any cycle of \\(G\\), a bipartite graph contains no odd cycle.\nProof (\\(\\Leftarrow\\)): Now suppose that \\(G\\) contains no odd cycle. We may assume \\(G\\) is connected: if it is not, we build a bipartition for each connected component and join the corresponding sides, since no edge links different components. We will construct a bipartition of \\(G\\) explicitly.\nLet \\(v\\) be any vertex of \\(G\\), and define: $$V_1 = \\{w \\in V(G) : d(v,w) \\text{ is even}\\} \\qquad V_2 = \\{u \\in V(G) : d(v,u) \\text{ is odd}\\}$$Since \\(G\\) is connected, every vertex is at a finite distance from \\(v\\), even or odd (never both), so we have \\(V_1 \\cup V_2 = V(G)\\) and \\(V_1 \\cap V_2 = \\emptyset\\). It remains to show that \\((V_1, V_2)\\) is indeed a bipartition, that is, that there is no edge inside \\(V_1\\) nor inside \\(V_2\\).\nSuppose, for contradiction, that this is false: there is an edge \\((w_i, w_j)\\) with \\(w_i, w_j \\in V_1\\) (the case of an edge inside \\(V_2\\) is analogous). Let \\(P_i\\) and \\(P_j\\) be the shortest paths from \\(v\\) to \\(w_i\\) and from \\(v\\) to \\(w_j\\), respectively, both of even length, since \\(w_i, w_j \\in V_1\\). Let \\(z\\) be the vertex common to \\(P_i\\) and \\(P_j\\) that is farthest from \\(v\\) (at least \\(v\\) itself is in common). The figure below sketches this construction:\nGeometric construction of the proof: v, z, Pi, Pj, and the edge that closes the odd cycle The subpath of \\(P_i\\) that goes from \\(v\\) to \\(z\\), denoted \\(P_i(v{-}z)\\), has the same length as the subpath \\(P_j(v{-}z)\\); otherwise, one of the two paths would not be the shortest to its respective destination (it would suffice to swap the longer stretch for the shorter stretch of the other path to obtain a shortcut). Since \\(P_i\\) and \\(P_j\\) have even length in total, and the parts from \\(v\\) to \\(z\\) have the same length, the remaining subpaths \\(P_i(z{-}w_i)\\) and \\(P_j(z{-}w_j)\\) have the same parity as each other (both even or both odd).\nBut the edge \\((w_i, w_j)\\) exists by hypothesis. Hence the route \\(P_i(z{-}w_i) \\cup (w_i,w_j) \\cup P_j(z{-}w_j)\\), which goes from \\(z\\) to \\(w_i\\), then to \\(w_j\\) through the edge, and back to \\(z\\) along \\(P_j\\), forms a cycle of length \\(\\text{len}(P_i(z{-}w_i)) + 1 + \\text{len}(P_j(z{-}w_j))\\). Since the two terms have the same parity, this sum is always odd (even + even + 1, or odd + odd + 1). This produces an odd cycle in \\(G\\), contradicting the hypothesis that \\(G\\) contains no odd cycle.\nThis contradiction shows that the edge \\((w_i, w_j)\\) cannot exist (and, analogously, no edge inside \\(V_2\\) can exist). Hence \\((V_1, V_2)\\) is indeed a bipartition of \\(G\\), that is, \\(G\\) is bipartite.\nTherefore, with both directions, a graph is bipartite if and only if it contains no cycle of odd length, which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary Every cycle \\(C_n\\) with \\(n\\) even is bipartite. Every cycle \\(C_n\\) with \\(n\\) odd is not bipartite.\nApplication: the proof becomes an algorithm The construction used in the proof (\\(\\Leftarrow\\)), which colors each vertex by the parity of its distance to a reference vertex, is the algorithm that graph libraries use to test bipartiteness. Implementing the same idea with the breadth-first search (BFS) we saw in the section on distances:\nfrom collections import deque import networkx as nx def bipartition(G, source): \u0026#34;\u0026#34;\u0026#34;V1 = even distance from source, V2 = odd distance: the same construction used in the proof of the theorem.\u0026#34;\u0026#34;\u0026#34; color = {source: 0} queue = deque([source]) while queue: v = queue.popleft() for neighbor in G.neighbors(v): if neighbor not in color: color[neighbor] = 1 - color[v] queue.append(neighbor) elif color[neighbor] == color[v]: return None # edge inside the same side: not bipartite V1 = sorted(v for v, c in color.items() if c == 0) V2 = sorted(v for v, c in color.items() if c == 1) return V1, V2 print(\u0026#34;C6:\u0026#34;, bipartition(nx.cycle_graph(6), 0)) print(\u0026#34;C5:\u0026#34;, bipartition(nx.cycle_graph(5), 0)) C6: ([0, 2, 4], [1, 3, 5]) C5: None For \\(C_6\\) (an even cycle), the function returns exactly the expected alternating bipartition: even-position vertices on one side, odd on the other, as the figure below shows. For \\(C_5\\) (an odd cycle), BFS eventually finds an edge between two vertices of the same color (the same contradiction exploited in the proof), and the function returns None, confirming that no bipartition exists.\nC6 bipartitioned by the BFS algorithm The networkx library already ships this check ready-made as nx.is_bipartite, which agrees with our implementation:\nimport networkx as nx print(nx.is_bipartite(nx.cycle_graph(6))) # True print(nx.is_bipartite(nx.cycle_graph(5))) # False Application: revisiting Zachary\u0026rsquo;s Karate Club Zachary\u0026rsquo;s Karate Club graph, the 34-member social network that we already saw when studying vertex degree, is a good test of this article\u0026rsquo;s properties on a real graph, much larger than the examples we solved by hand:\nimport networkx as nx G = nx.karate_club_graph() print(\u0026#34;Connected:\u0026#34;, nx.is_connected(G)) print(\u0026#34;Diameter:\u0026#34;, nx.diameter(G)) print(\u0026#34;Center:\u0026#34;, nx.center(G)) print(\u0026#34;Bipartite:\u0026#34;, nx.is_bipartite(G)) Connected: True Diameter: 5 Center: [0, 1, 2, 3, 8, 13, 19, 31] Bipartite: False The whole club is a single connected component (\\(\\omega(G)=1\\)), with diameter 5, a little larger than that of the graph \\(G\\) in this article, but still small for a real social network. Note that vertex \\(33\\) (the club administrator, of highest degree, with \\(17\\) connections, as we saw in the article on degree) is not in the center: its eccentricity is \\(4\\), versus \\(3\\) for vertex \\(0\\). Having more connections does not guarantee being more \u0026ldquo;in the middle\u0026rdquo; of the network in terms of distance: these are different notions of centrality.\nAnd the graph is not bipartite: a real friendship network has groups that form triangles (\\(A\\) and \\(B\\) are friends, \\(B\\) and \\(C\\) are friends, and \\(A\\) and \\(C\\) end up becoming friends too), and every triangle is an odd cycle, which is enough, by the characterization theorem, to rule out any chance of a bipartition.\nExercises # The following exercises apply the concepts of walk, trail, path, and bipartition seen in this article, in increasing order of difficulty.\nExercise 1: Classifying sequences Statement: Classify each sequence below as a walk, trail, path, or none of these, in a graph with vertices \\(\\{1,2,3,4,5\\}\\) and edges \\(\\{1,2\\}, \\{2,3\\}, \\{3,4\\}, \\{4,5\\}, \\{1,3\\}, \\{3,5\\}\\):\n(a) \\(1, 2, 3, 4, 5, 3, 1\\) (b) \\(1, 3, 2, 4\\) (c) \\(1, 2, 3, 5, 4\\) (d) \\(1, 3, 5, 4, 3, 2\\)\nSolution (a) It is a (closed) trail, because the sequence traverses all 6 edges of the graph, each exactly once: \\(\\{1,2\\}, \\{2,3\\}, \\{3,4\\}, \\{4,5\\}, \\{3,5\\}, \\{1,3\\}\\). Since no edge repeats, it is a trail, and since \\(v_0 = v_6 = 1\\), it is closed. It is neither a path nor a cycle, since vertex 3 is visited twice (at positions 3 and 6).\n(b) It is not valid: there is no edge \\(\\{2,4\\}\\).\n(c) It is a path, because no vertex repeats and all the edges exist: \\(\\{1,2\\}, \\{2,3\\}, \\{3,5\\}, \\{5,4\\}\\). ✓\n(d) It is a trail: no edge repeats, but vertex 3 appears twice.\nThe graph used in the exercise, to visualize the sequences above:\nGraph of Exercise 1 Exercise 2: Bipartite? Statement: Determine whether the cycle \\(C_6\\) is bipartite and, if it is, give the bipartition.\nSolution \\(C_6\\) has even length, so it is bipartite by the characterization theorem. For \\(C_6 = v_1-v_2-v_3-v_4-v_5-v_6-v_1\\), the bipartition is \\(V_1 = \\{v_1, v_3, v_5\\}\\) and \\(V_2 = \\{v_2, v_4, v_6\\}\\). \\(\\blacksquare\\)\nSummary # Concept Description Walk Sequence of vertices \\(v_0, \\ldots, v_k\\) with \\((v_i,v_{i+1}) \\in E\\); repetitions allowed Trail Walk with no repeated edges Path Walk with no repeated vertices Closed trail Closed walk (\\(v_0=v_k\\)) with no repeated edges Cycle Closed walk with no repeated internal vertices Even / odd cycle Cycle with an even / odd number of edges Connected vertices A path exists between them Connected Every pair of vertices is connected Connected component Connected, maximal induced subgraph (partition of \\(V\\)) Distance \\(d(v,w)\\) Length of the shortest path; \\(\\infty\\) if \\(v,w\\) are disconnected Breadth-first search (BFS) Computes distances from a source in \\(O(n+m)\\) Dijkstra\u0026rsquo;s algorithm Distances in graphs with nonnegative weights, with a priority queue Eccentricity \\(e(v)\\) Greatest distance from \\(v\\) to any other vertex Diameter Greatest eccentricity of the graph (assumes \\(G\\) connected) Center \\(c(G)\\) Vertices of smallest eccentricity Trivial / null graph \\(K_1\\) (one vertex, no edges) / \\(N_n\\) (\\(n\\) vertices, no edges) Bipartite \\(V = V_1 \\cup V_2\\), \\(V_1 \\cap V_2 = \\emptyset\\), edges only between \\(V_1\\) and \\(V_2\\) Bipartite ↔ No cycles of odd length Next Steps # We have answered the opening question: we now know how to recognize whether two points of a graph are connected, measure the shortest distance between them, and decide whether the whole structure is bipartite. But walks, cycles, and connectivity do not answer this question: what is the smallest number of edges capable of keeping a graph connected, with no leftover cycle? That is the question that motivates trees, the topic of the next article in the series.\n","date":"2026-08-13","externalUrl":null,"permalink":"/2026/08/paths-cycles/","section":"Blog","summary":"How do we know whether two points of a network are connected, and by which shortest path? We define walk, trail, path, and cycle, connectivity and distance, compute distances with breadth-first search and Dijkstra, and prove that a graph is bipartite if and only if it has no odd cycle.","title":"Paths, Cycles, and Connectivity","type":"posts"},{"content":" Two graphs can have vertices with different names and still be, in essence, exactly the same. Imagine a subway map drawn in two different ways: the stations have different labels on paper, but the connections between them are the same. Mathematically, these two graphs are isomorphic.\nIn this article, we formalize this notion and also study how to represent graphs so that computers can work with them: adjacency matrices and incidence matrices.\nWhy Do Isomorphism and Matrix Representation Matter? # Chemistry: two molecules can be described by different graphs (atoms numbered in different ways) and still be the same substance, and deciding that is deciding a graph isomorphism. Circuit verification: two circuit schematics drawn in different ways can represent exactly the same network of components. Compilers and optimization: equivalent control-flow graphs, produced by different compilation paths, must be recognized as \u0026ldquo;the same graph\u0026rdquo; so that the same optimizations can be applied. Data structures in software: graph libraries must choose how to store a graph in memory. networkx, in Python, uses adjacency lists (dictionaries of neighbors) and produces the adjacency or incidence matrix when asked; numerical libraries work directly with matrices. Understanding these representations is understanding how the computer really \u0026ldquo;sees\u0026rdquo; a graph. Linear algebra applied to networks: eigenvalues and eigenvectors of the adjacency matrix (the \u0026ldquo;spectrum\u0026rdquo; of the graph) are used in algorithms such as PageRank and in social network analysis. Graph Isomorphism # Same Graph, Different Drawings # Before defining isomorphism formally, it is worth seeing the phenomenon in practice: very different geometric representations can correspond to exactly the same graph (same vertex set, same edge set).\nExample: two representations of the same graph Statement: Let \\(G\\) be the graph with \\(V(G) = \\{a,b,c,d,e\\}\\) and \\(E(G) = \\{(a,b), (b,c), (a,c), (a,e), (c,d), (e,d)\\}\\). Do the two geometric representations below show the same graph?\nSame graph, two geometric representations Solution: Yes. The two figures have completely different vertex positions, but they describe exactly the same graph: same vertices, same edges.\nExample: \\(K_4\\) in three representations Statement: The complete graph \\(K_4\\) (vertices \\(\\{1,2,3,4\\}\\), all \\(\\binom{4}{2} = 6\\) possible edges) can be drawn in visually very different ways. Do the three figures below represent the same graph?\nK4 in three geometric representations Solution: Yes. At first sight the three figures look like different graphs, but all of them represent \\(K_4\\).\nSimilar Drawings, Different Graphs # The converse also happens: two drawings can look practically identical and still correspond to formally different graphs.\nExample: \\(H_1\\) and \\(H_2\\) Statement: Let \\(H_1\\) and \\(H_2\\) be two graphs with the same vertex set \\(V(H_1) = V(H_2) = \\{1,2,a,b\\}\\), but with\n$$E(H_1) = \\{(1,a), (1,b), (2,a), (2,b)\\}$$$$E(H_2) = \\{(1,a), (1,2), (b,a), (b,2)\\}$$Are \\(H_1\\) and \\(H_2\\) the same graph? Are they isomorphic?\nH1 and H2: similar drawings, different graphs Solution: The two drawings have the same visual crossing pattern, but \\(E(H_1) \\neq E(H_2)\\) as sets of pairs with the same labels, so \\(H_1\\) and \\(H_2\\) are formally different graphs.\nCuriously, if we rename the vertices of \\(H_2\\) by swapping \\(b\\) with \\(2\\) and \\(2\\) with \\(b\\), we obtain exactly \\(H_1\\). That is, \\(H_1\\) and \\(H_2\\) are not the same graph, but they are isomorphic to each other, a notion we will formalize next.\nThe Isomorphism Problem # The examples above raise a natural question: given two geometric representations of graphs, do they correspond to the same graph (up to renaming vertices)? In other words, is it possible to associate each point of one representation with a point of the other so as to preserve adjacencies, that is, so that the edges coincide?\nThis question has a name: the Graph Isomorphism Problem. The formal definition below is the precise answer to this question.\nIsomorphism Two graphs \\(G_1\\) and \\(G_2\\) are isomorphic, denoted \\(G_1 \\approx G_2\\), when there exists a bijective function (injective and surjective) $$f: V(G_1) \\to V(G_2)$$ such that: $$(v, w) \\in E(G_1) \\iff (f(v), f(w)) \\in E(G_2)$$In words: \\(f\\) is a bijection between the vertices that preserves adjacencies, that is, two vertices are adjacent in \\(G_1\\) if and only if their images are adjacent in \\(G_2\\). The function \\(f\\) is called an isomorphism between \\(G_1\\) and \\(G_2\\).\nNecessary (But Not Sufficient) Conditions # From the definition itself we can already deduce mandatory consequences of \\(G_1 \\approx G_2\\). For \\(G_1 \\approx G_2\\), the following conditions are necessary:\n\\(|V(G_1)| = |V(G_2)|\\) (same number of vertices), a direct consequence of \\(f\\) being injective and surjective. \\(|E(G_1)| = |E(G_2)|\\) (same number of edges), because for each pair \\((v,w) \\in E(G_1)\\) there is a corresponding pair \\((f(v), f(w)) \\in E(G_2)\\), and since \\(f\\) is bijective this correspondence is also a bijection between the edges. The degree sequences of \\(G_1\\) and \\(G_2\\) are equal: since \\(f\\) preserves adjacencies vertex by vertex, the degree of \\(v\\) in \\(G_1\\) equals the degree of \\(f(v)\\) in \\(G_2\\). Necessary conditions, not sufficient The conditions above are necessary, but not sufficient. Two graphs can satisfy all of them and still not be isomorphic, as the counterexample below shows.\nThe problem of deciding whether two graphs are isomorphic (the Graph Isomorphism Problem) is one of the great open problems of computational complexity theory: it is not known whether it is in P (the class of problems solvable in polynomial time), but it has not been proved NP-complete either. A problem is NP-complete when it is among the hardest in the class NP (short for nondeterministic polynomial time, the class of problems whose solution, once found, can be verified quickly): every other problem in NP can be reduced to it in polynomial time, so that if someone found a fast algorithm for an NP-complete problem, they would have a fast algorithm for all problems in NP.\nThis undefined status is rare among natural problems in NP. In 2015, László Babai announced a quasipolynomial-time algorithm, of the order of \\(2^{O((\\log n)^c)}\\) for some constant \\(c\\), much faster than any exponential algorithm, although still not polynomial. In January 2017, Harald Helfgott found a flaw in the proof; Babai fixed it within a few days and the result stood (see the entry Graph isomorphism problem).\nWhen the Necessary Conditions Are Not Enough # Let us start with the easy case: when the necessary conditions already fail, the answer is immediate.\nExample: non-isomorphic graphs (easy case) Statement: Let \\(G_1\\) be the path \\(P_4\\), with degree sequence \\((1,1,2,2)\\), and \\(G_2\\) the star graph \\(K_{1,3}\\), with degree sequence \\((1,1,1,3)\\). Are \\(G_1\\) and \\(G_2\\) isomorphic?\nSolution: No. Both have 4 vertices and 3 edges, but the degree sequences are different. Hence \\(G_1 \\not\\approx G_2\\).\nNow the hard case, the reason the warning box above exists: two graphs can satisfy all the necessary conditions seen so far and still not be isomorphic.\nCounterexample: when the necessary conditions mislead Statement: Let \\(G_1\\) and \\(G_2\\) be the graphs below.\nG1 and G2: necessary conditions are not enough $$V(G_1) = \\{1,2,3,4,5,6,7,8\\}$$ $$\\begin{aligned}E(G_1) = \\{\u0026(1,2),(2,3),(3,4),(1,4),(8,7),\\\\ \u0026(7,5),(5,6),(6,8),(1,8),(3,5)\\}\\end{aligned}$$$$V(G_2) = \\{a,b,c,d,e,f,g,h\\}$$ $$\\begin{aligned}E(G_2) = \\{\u0026(a,b),(b,c),(c,d),(d,a),(g,h),\\\\ \u0026(h,f),(f,e),(e,g),(a,g),(b,h)\\}\\end{aligned}$$Are \\(G_1\\) and \\(G_2\\) isomorphic?\nSolution: First, we check the necessary conditions: \\(|V(G_1)| = |V(G_2)| = 8\\), \\(|E(G_1)| = |E(G_2)| = 10\\), and the degree sequences of both are \\((2,2,2,2,3,3,3,3)\\). Everything matches! We are tempted to conclude that \\(G_1 \\approx G_2\\).\nBut looking more carefully at how the degree-3 vertices connect to each other:\nIn \\(G_1\\), vertex \\(1\\) (degree 3) is adjacent to \\(2\\) (degree 2), \\(4\\) (degree 2) and \\(8\\) (degree 3), and each degree-3 vertex is adjacent to two degree-2 vertices and one degree-3 vertex. In \\(G_2\\), vertex \\(a\\) (degree 3) is adjacent to \\(b\\) (degree 3), \\(d\\) (degree 2) and \\(g\\) (degree 3), and each degree-3 vertex is adjacent to two degree-3 vertices and one degree-2 vertex, the opposite pattern! Since this local adjacency structure is different, no bijection can preserve all adjacencies simultaneously. Hence \\(G_1 \\not\\approx G_2\\), even though they satisfy all the necessary conditions seen so far. This is exactly what it means to say that these conditions are not sufficient.\nProving an Isomorphism # After seeing that checking the necessary conditions can be misleading, the only reliable way to guarantee that \\(G_1 \\approx G_2\\) is to explicitly exhibit a bijection \\(f\\) and verify that it preserves all adjacencies.\nExample: proving \\(G_1 \\approx G_2\\) Statement: Let \\(G_1\\) and \\(G_2\\) be the graphs below, given by their geometric representation.\nG1 and G2: proof of isomorphism via a bijection $$V(G_1) = \\{1,2,3,4,5,6,7,8\\}$$ $$\\begin{aligned}E(G_1) = \\{\u0026(1,2),(2,3),(3,4),(1,4),(5,6),(6,7),\\\\ \u0026(7,8),(5,8),(1,5),(2,6),(3,7),(4,8)\\}\\end{aligned}$$$$V(G_2) = \\{a,b,c,d,e,f,g,h\\}$$ $$\\begin{aligned}E(G_2) = \\{\u0026(a,b),(b,h),(h,g),(g,a),(c,d),(d,f),\\\\ \u0026(f,e),(e,c),(a,c),(b,d),(g,e),(h,f)\\}\\end{aligned}$$Are \\(G_1\\) and \\(G_2\\) isomorphic?\nSolution: Before trying to prove that they are isomorphic, it is worth checking the necessary conditions: \\(|V(G_1)| = |V(G_2)| = 8\\), \\(|E(G_1)| = |E(G_2)| = 12\\), and both are 3-regular (every vertex has degree 3). The conditions match, but this only tells us that an isomorphism may exist, not that it does.\nTo prove that it exists, we exhibit the bijection \\(f: V(G_1) \\to V(G_2)\\):\n\\(v\\) \\(f(v)\\) 1 a 2 c 3 d 4 b 5 g 6 e 7 f 8 h It suffices to check that each of the 12 edges of \\(G_1\\) maps to an actual edge of \\(G_2\\). For example: \\((1,2) \\in E(G_1)\\) and \\((f(1),f(2)) = (a,c) \\in E(G_2)\\); \\((1,4) \\in E(G_1)\\) and \\((f(1),f(4)) = (a,b) \\in E(G_2)\\); \\((1,5) \\in E(G_1)\\) and \\((f(1),f(5)) = (a,g) \\in E(G_2)\\); and so on for the remaining edges. All of them check out, hence \\(G_1 \\approx G_2\\).\nWhy not test all possible bijections? The most natural procedure to decide whether two graphs with \\(n\\) vertices are isomorphic would be brute force: examine each of the \\(n!\\) possible permutations of vertices (that is, test each candidate bijective function) until finding one that preserves adjacencies, or exhausting all of them without success. The problem is that \\(n!\\) grows very fast: for \\(n = 20\\) there are already more than \\(2 \\times 10^{18}\\) permutations. That is why the Graph Isomorphism Problem is considered computationally hard in practice, even without a definite status in complexity theory.\nAdjacency Matrix # For computer algorithms to process graphs, we need numerical representations. The most natural one is the adjacency matrix.\nAdjacency Matrix Given a graph \\(G\\) with \\(n\\) vertices \\(V = \\{v_1, v_2, \\ldots, v_n\\}\\), the adjacency matrix \\(A = (a_{ij})\\) is an \\(n \\times n\\) matrix where: $$a_{ij} = \\begin{cases} 1 \u0026 \\text{if } (v_i, v_j) \\in E(G) \\\\ 0 \u0026 \\text{otherwise} \\end{cases}$$ Example: adjacency matrix of a graph with 5 vertices Statement: Write the adjacency matrix of the graph \\(G\\) with\n$$V(G) = \\{v_1, v_2, v_3, v_4, v_5\\}$$ $$E(G) = \\{(v_1,v_2), (v_1,v_3), (v_1,v_5), (v_2,v_3), (v_3,v_4), (v_4,v_5)\\}$$Graph with 5 vertices and 6 edges Solution: $$A = \\begin{pmatrix} 0 \u0026 1 \u0026 1 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\\\ 1 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \\\\ 0 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \\end{pmatrix}$$Properties of the adjacency matrix of a simple graph:\nThe main diagonal is all zeros: \\(a_{ii} = 0\\) (no loops) The matrix is symmetric: \\(a_{ij} = a_{ji}\\) (since edges are unordered pairs, in undirected graphs) The number of 1\u0026rsquo;s equals \\(2m\\), where \\(m = |E(G)|\\) (each edge \\((v_i,v_j)\\) contributes a 1 at \\(a_{ij}\\) and another at \\(a_{ji}\\)) An adjacency matrix uniquely characterizes a graph, but the same graph can have several different adjacency matrices: just permute the order in which the vertices are listed. Example: permuting the order of the vertices Statement: Reorder the vertices of the graph in the previous example from \\((v_1,v_2,v_3,v_4,v_5)\\) to \\((v_3,v_2,v_5,v_1,v_4)\\) and write the new adjacency matrix.\nSolution: $$A' = \\begin{pmatrix} 0 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \\\\ 1 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \\\\ 0 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \\\\ 1 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \\\\ 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\end{pmatrix}$$Visually, \\(A\\) and \\(A^{\\prime}\\) are different matrices, but both represent exactly the same graph.\nCuriosity: counting walks The adjacency matrix holds more information than it seems: the entry \\((A^k)_{ij}\\) of the \\(k\\)-th power of \\(A\\) equals the number of walks of length \\(k\\) between \\(v_i\\) and \\(v_j\\). It is one of the reasons this representation is used in graph algorithms.\nIncidence Matrix # An alternative representation is the incidence matrix, which relates vertices to edges.\nIncidence Matrix Given a graph \\(G\\) with \\(n\\) vertices and \\(m\\) edges, the incidence matrix \\(B = (b_{ij})\\) is an \\(n \\times m\\) matrix where: $$b_{ij} = \\begin{cases} 1 \u0026 \\text{if vertex } v_i \\text{ is an endpoint of edge } e_j \\\\ 0 \u0026 \\text{otherwise} \\end{cases}$$ Example: incidence matrix of the same graph Statement: Write the incidence matrix of the 5-vertex graph from the previous example, now with labeled edges: \\(e_1=(v_1,v_2)\\), \\(e_2=(v_1,v_3)\\), \\(e_3=(v_2,v_3)\\), \\(e_4=(v_3,v_4)\\), \\(e_5=(v_4,v_5)\\), \\(e_6=(v_1,v_5)\\).\nGraph with 5 vertices and labeled edges Solution: $$B = \\begin{array}{c|cccccc} \u0026 e_1 \u0026 e_2 \u0026 e_3 \u0026 e_4 \u0026 e_5 \u0026 e_6 \\\\ \\hline v_1 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \\\\ v_2 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \\\\ v_3 \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \\\\ v_4 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \\\\ v_5 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \\end{array}$$Row sums: \\(d(v_1)=3, d(v_2)=2, d(v_3)=3, d(v_4)=2, d(v_5)=2\\). Sum of all entries: \\(12 = 2 \\times 6\\) edges.\nProperties of the incidence matrix of a simple graph:\nEach column has exactly two 1s (each edge has two endpoints) The sum of row \\(i\\) equals \\(d(v_i)\\) (the degree of the vertex) The sum of all entries is \\(2|E|\\), which confirms the Handshaking Theorem An incidence matrix uniquely characterizes a graph, but the same graph can have several different incidence matrices: just permute the order of the vertices and/or the order of the edges. Example: permuting vertices and edges Statement: Simultaneously permute \\((v_1,v_2,v_3,v_4,v_5) \\to (v_3,v_5,v_1,v_4,v_2)\\) and \\((e_1,\\ldots,e_6) \\to (e_5,e_3,e_2,e_1,e_6,e_4)\\) and write the new incidence matrix.\nSolution: $$B' = \\begin{array}{c|cccccc} \u0026 e_5 \u0026 e_3 \u0026 e_2 \u0026 e_1 \u0026 e_6 \u0026 e_4 \\\\ \\hline v_3 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 1 \\\\ v_5 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \\\\ v_1 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 0 \\\\ v_4 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \\\\ v_2 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\end{array}$$Rows and columns in a different order, but still the same graph, exactly as happened with the adjacency matrix.\nThe Isomorphism Problem via Matrices # Now that we know how to represent graphs numerically, the Isomorphism Problem can be restated in terms of matrices: given two \\(n \\times n\\) adjacency matrices (or two \\(n \\times m\\) incidence matrices), do they represent the same graph? It is the same question from the beginning of the article (remember the subway map redrawn with different labels?), only now applied to numerical representations instead of geometric ones. And it suffers from exactly the same difficulty: it is not enough to compare the matrices entry by entry, we must find out whether there is some permutation of rows and columns that turns one into the other.\nSummary Table # The table below summarizes the central definitions, conditions and properties presented in this article.\nConcept Notation / Formula Description Isomorphism \\(G_1 \\approx G_2\\) if \\(\\exists f: V(G_1) \\to V(G_2)\\) bijective with \\((v,w) \\in E(G_1) \\iff (f(v),f(w)) \\in E(G_2)\\) \\(f\\) preserves adjacencies Necessary conditions \\(\\lvert V(G_1)\\rvert = \\lvert V(G_2)\\rvert\\), \\(\\lvert E(G_1)\\rvert = \\lvert E(G_2)\\rvert\\), same degree sequence Necessary, not sufficient Adjacency matrix \\(A = (a_{ij})\\), \\(n \\times n\\), \\(a_{ij} = 1\\) if \\((v_i,v_j) \\in E\\) Zero diagonal, symmetric, number of 1\u0026rsquo;s \\(= 2m\\), not unique (vertex permutation) Incidence matrix \\(B = (b_{ij})\\), \\(n \\times m\\), \\(b_{ij} = 1\\) if \\(v_i\\) is an endpoint of \\(e_j\\) Each column with two 1\u0026rsquo;s, sum of row \\(i\\) \\(= d(v_i)\\), total sum \\(= 2\\lvert E\\rvert\\), not unique (vertex and edge permutation) Powers of the adjacency matrix \\((A^k)_{ij}\\) Number of walks of length \\(k\\) between \\(v_i\\) and \\(v_j\\) Comparing the Representations # Besides knowing how to build each matrix, it is worth directly comparing the practical cost of using them in an algorithm:\nAspect Adjacency Matrix Incidence Matrix Dimension \\(n \\times n\\) \\(n \\times m\\) Symmetry Symmetric (undirected graph) Not applicable (matrix is not square) Check adjacency \\(O(1)\\) \\(O(m)\\) List neighbors of \\(v\\) \\(O(n)\\) \\(O(m)\\) Memory usage \\(O(n^2)\\) \\(O(nm)\\) For dense graphs (many edges), the adjacency matrix is usually more efficient. For sparse graphs, representations that store only the neighbors of each vertex (the so-called adjacency lists, one list per vertex instead of a full \\(n \\times n\\) matrix of zeros) are preferable in practice.\nExercises # Exercise 1: Isomorphism Are the graphs \\(G_1\\) (a triangle plus an isolated vertex: \\(K_3 \\cup K_1\\)) and \\(G_2\\) (the path \\(P_4\\)) isomorphic?\nSolution No. The degree sequence of \\(G_1\\) is \\((0, 2, 2, 2)\\) and that of \\(G_2\\) is \\((1, 1, 2, 2)\\). Since the sequences are different, \\(G_1 \\not\\approx G_2\\). \\(\\blacksquare\\)\nExercise 2: Adjacency matrix Write the adjacency matrix of the cycle \\(C_5\\) with vertices \\(\\{1, 2, 3, 4, 5\\}\\) and edges \\((1,2), (2,3), (3,4), (4,5), (5,1)\\).\nSolution $$A = \\begin{pmatrix} 0 \u0026 1 \u0026 0 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\\\ 0 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \\\\ 0 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \\end{pmatrix}$$All vertices have degree 2 (each row sums to 2), confirming that \\(C_5\\) is 2-regular. \\(\\blacksquare\\)\nExercise 3: Incidence matrix Let \\(G\\) be the graph \u0026ldquo;triangle with a pendant edge\u0026rdquo;: \\(V(G) = \\{p,q,r,s\\}\\), \\(E(G) = \\{(p,q), (q,r), (p,r), (r,s)\\}\\). Write the incidence matrix \\(B\\) of \\(G\\) and confirm that the sum of each row matches the degree of the corresponding vertex.\nSolution Labeling \\(e_1=(p,q)\\), \\(e_2=(q,r)\\), \\(e_3=(p,r)\\), \\(e_4=(r,s)\\):\n$$B = \\begin{array}{c|cccc} \u0026 e_1 \u0026 e_2 \u0026 e_3 \u0026 e_4 \\\\ \\hline p \u0026 1 \u0026 0 \u0026 1 \u0026 0 \\\\ q \u0026 1 \u0026 1 \u0026 0 \u0026 0 \\\\ r \u0026 0 \u0026 1 \u0026 1 \u0026 1 \\\\ s \u0026 0 \u0026 0 \u0026 0 \u0026 1 \\end{array}$$Row sums: \\(d(p)=2, d(q)=2, d(r)=3, d(s)=1\\), and they match exactly the degrees of each vertex in \\(G\\). \\(\\blacksquare\\)\nExercise 4: Isomorphism via matrices The adjacency matrices below represent, respectively, the graphs \\(G_1\\) (vertices \\(1,2,3,4\\)) and \\(G_2\\) (vertices \\(a,b,c,d\\)):\n$$A_1 = \\begin{pmatrix} 0 \u0026 1 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 1 \u0026 0 \\\\ 0 \u0026 1 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 1 \u0026 0 \\end{pmatrix} \\qquad A_2 = \\begin{pmatrix} 0 \u0026 1 \u0026 1 \u0026 0 \\\\ 1 \u0026 0 \u0026 0 \u0026 1 \\\\ 1 \u0026 0 \u0026 0 \u0026 1 \\\\ 0 \u0026 1 \u0026 1 \u0026 0 \\end{pmatrix}$$Are \\(G_1\\) and \\(G_2\\) isomorphic?\nSolution Yes, although they are not the same graph (the vertices have different names). \\(A_1\\) describes the cycle \\(1-2-3-4-1\\), and \\(A_2\\) describes the cycle \\(a-b-d-c-a\\) (check: \\(a\\) is adjacent to \\(b\\) and \\(c\\); \\(b\\) is adjacent to \\(a\\) and \\(d\\); etc.). The bijection \\(f(1)=a, f(2)=b, f(3)=d, f(4)=c\\) preserves all adjacencies, hence \\(G_1 \\approx G_2\\). \\(\\blacksquare\\)\nNext Steps # Isomorphism and matrices answer one specific question: given two entire graphs, are they \u0026ldquo;the same\u0026rdquo;? But neither of these tools tells us how to move within a single graph: what the route between two vertices is, when that route forms a cycle, or whether the whole graph is connected. These are the questions of the next article in the series: paths, cycles and connectivity.\n","date":"2026-08-11","externalUrl":null,"permalink":"/2026/08/graph-isomorphism-matrices/","section":"Blog","summary":"Two graphs with the same number of vertices, the same number of edges and the same degree sequence may not be isomorphic. We will see what isomorphism is, how to prove it with a bijection, and how to represent graphs by adjacency and incidence matrices.","title":"Isomorphism and Matrix Representation","type":"posts"},{"content":" One of the simplest and, at the same time, most useful notions in graph theory is the degree of a vertex, that is, how many edges are connected to it. From this seemingly trivial definition emerges a central result of the field: the Handshaking Theorem.\nWhy Does the Degree of a Vertex Matter? # The degree of a vertex is a simple local measure, yet it appears behind important results in mathematics and computing:\nSocial networks: the degree of a vertex directly measures the number of connections of a person (or profile), the basis of centrality metrics and of \u0026ldquo;hub\u0026rdquo; detection in a network. Routing and infrastructure: in a computer or transportation network, the degree of a node indicates how many alternative routes pass through it, which is important information for analyzing robustness and single points of failure. Circuit and hardware design: the degree of a vertex in a connection graph limits the number of ports or physical pins needed at that point of the circuit. Structure checking: as we will see in the Handshaking Theorem, the sum of the degrees of a graph is always equal to twice the number of edges, a quick and cheap consistency check for any graph structure before running more costly algorithms on it. Degree of a Vertex # Degree of a Vertex Let \\(G = (V, E)\\) be a graph and \\(v \\in V\\). The degree of \\(v\\), denoted \\(d(v)\\) or \\(d_G(v)\\) (when more than one graph is in play), is the number of edges incident to \\(v\\), that is, the number of vertices adjacent to \\(v\\): $$d(v) = |N(v)| = |\\{w \\in V \\mid (v, w) \\in E\\}|$$ Note In multigraphs with loops (edges from a vertex to itself), the usual convention is that a loop contributes 2 to the degree of the vertex, since the loop is incident twice to the vertex.\nExample 1 Problem: Let \\(G = (V, E)\\) with $$V = \\{x,\\ y,\\ z,\\ v,\\ w,\\ t,\\ u\\}$$ $$E = \\{(x,y),\\ (y,z),\\ (x,z),\\ (x,w),\\ (x,v),\\ (z,v),\\ (w,v),\\ (z,t)\\}$$ Compute the degree of each vertex of \\(G\\).\nSolution: Counting the edges incident to each vertex:\n\\(d(x) = 4\\) (neighbors \\(y, z, w, v\\)) \\(d(y) = 2\\) (neighbors \\(x, z\\)) \\(d(z) = 4\\) (neighbors \\(y, x, v, t\\)) \\(d(v) = 3\\) (neighbors \\(x, z, w\\)) \\(d(w) = 2\\) (neighbors \\(x, v\\)) \\(d(t) = 1\\) (neighbor \\(z\\)) \\(d(u) = 0\\) (no neighbors: isolated vertex) The figure below shows \\(G\\) with the degree of each vertex annotated. This is the graph we will reuse in the next examples.\nExample 1: graph G and its degrees Example 2 Problem: Let \\(G\\) be the multigraph with \\(V = \\{x, y, z, v, w\\}\\), in which there is a loop at vertex \\(z\\), two parallel edges between \\(x\\) and \\(y\\), and three between \\(w\\) and \\(v\\) (see the figure): $$E = \\{(x,y),\\ (x,y),\\ (y,z),\\ (y,v),\\ (x,v),\\ (x,w),\\ (z,v),\\ (w,v),\\ (w,v),\\ (w,v),\\ (z,z)\\}$$ Compute the degree of each vertex, remembering that the loop at \\(z\\) contributes 2 units.\nSolution:\n\\(d(x) = 4\\) \\(d(y) = 4\\) \\(d(z) = 4\\) (includes the 2 units from the loop) \\(d(v) = 6\\) \\(d(w) = 4\\) The next figure shows the multigraph with the parallel edges (in orange) and the loop at \\(z\\) highlighted.\nExample 2: multigraph with a loop Minimum Degree and Maximum Degree # Minimum and Maximum Degree For a graph \\(G\\):\n\\(\\delta(G)\\) = minimum degree = smallest degree among all the vertices of \\(G\\) \\(\\Delta(G)\\) = maximum degree = largest degree among all the vertices of \\(G\\) Example 3 Problem: Consider again the graph \\(G\\) from Example 1. Determine \\(\\delta(G)\\) and \\(\\Delta(G)\\).\nSolution: The degrees computed in Example 1 were \\(d(x)=4\\), \\(d(y)=2\\), \\(d(z)=4\\), \\(d(v)=3\\), \\(d(w)=2\\), \\(d(t)=1\\), \\(d(u)=0\\). The smallest value is \\(0\\) (vertex \\(u\\)) and the largest is \\(4\\) (vertices \\(x\\) and \\(z\\)). Therefore: $$\\delta(G) = 0 \\qquad \\Delta(G) = 4$$The graph \\(G\\), with \\(u\\) (minimum degree) and \\(x\\), \\(z\\) (maximum degree) identifiable from the degree annotations.\nExample 3: δ(G) and Δ(G) Regular Graphs # Some graphs have all vertices with the same degree: these are the regular graphs.\n\\(k\\)-Regular Graph A graph \\(G\\) is called regular if all of its vertices have the same degree. In particular, if \\(d(v) = k\\) for every \\(v \\in V(G)\\), we say that \\(G\\) is \\(k\\)-regular or regular of degree \\(k\\).\nExample 4 Problem: Classify the graphs below by regularity.\n\\(G_1\\): three isolated vertices \\(a, b, c\\) \\(G_2\\): two pairs of vertices, each pair joined by one edge, \\((x,y)\\) and \\((z,w)\\) \\(G_3\\): a cycle of 4 vertices \\(1\\text{-}2\\text{-}3\\text{-}4\\text{-}1\\) Solution:\n\\(d(a) = d(b) = d(c) = 0\\) → \\(G_1\\) is 0-regular \\(d(x) = d(y) = d(z) = d(w) = 1\\) → \\(G_2\\) is 1-regular \\(d(1) = d(2) = d(3) = d(4) = 2\\) → \\(G_3\\) is 2-regular The three graphs are shown side by side below.\nExample 4: G1, G2, and G3 Example 4 (continued) Problem: Also classify the graphs \\(G_4\\) (the cube graph, with 8 vertices), \\(G_5 = K_4\\), and \\(G_6 = K_5\\).\nSolution:\n\\(G_4\\) (cube): every vertex has degree \\(3\\) → 3-regular \\(G_5 = K_4\\): every vertex has degree \\(3\\) → 3-regular \\(G_6 = K_5\\): every vertex has degree \\(4\\) → 4-regular The three graphs are shown below: the cube \\(G_4\\), \\(K_4\\), and \\(K_5\\).\nExample 4 (continued): G4, K4, and K5 Complete graphs are regular In general, \\(K_n\\) is regular of degree \\(n-1\\): each vertex is adjacent to all the other \\(n-1\\) vertices of the graph.\nSum of the Degrees of a Graph # Before stating the general result, it is worth summing the degrees of two concrete graphs and observing the pattern that emerges.\nExample 5 Problem: Sum the degrees of all the vertices of the graph \\(G\\) from Example 1.\nSolution: $$\\sum_{v \\in V} d(v) = d(x)+d(y)+d(z)+d(v)+d(w)+d(t)+d(u) = 4+2+4+3+2+1+0 = 16$$ The graph \\(G\\) has exactly \\(8\\) edges, and \\(16 = 2 \\times 8\\): the sum of the degrees is exactly twice the number of edges.\nExample 5: sum of the degrees of G Example 6 Problem: Sum the degrees of all the vertices of the cube graph \\(G_4\\) (Example 4), which has 8 vertices, all of degree 3.\nSolution: $$\\sum_{v \\in V(G_4)} d(v) = 3 \\times 8 = 24$$ The graph \\(G_4\\) has exactly \\(12\\) edges, and again \\(24 = 2 \\times 12\\).\nExample 6: sum of the degrees of the cube The two examples above suggest a general pattern, which we formalize next.\nThe Handshaking Theorem # The name of the following result comes from an analogy: if \\(n\\) people are in a room and each pair of people may or may not shake hands, the sum of the number of handshakes of each person is necessarily equal to twice the total number of handshakes (since each handshake is counted twice, once for each person involved).\nHandshaking Theorem For any graph \\(G = (V, E)\\): $$\\sum_{v \\in V} d(v) = 2|E|$$Proof:\nEach edge \\((u, v) \\in E\\) contributes exactly 1 to \\(d(u)\\) and exactly 1 to \\(d(v)\\). Hence, each edge is counted exactly 2 times in the sum \\(\\sum_{v \\in V} d(v)\\). Therefore, \\(\\sum_{v \\in V} d(v) = 2|E|\\), which is what we wanted to prove. \\(\\blacksquare\\)\nNote The Handshaking Theorem also holds for multigraphs (using the convention that each loop contributes 2 units to the degree of the vertex).\nIn practice: the Handshaking Theorem on a real social network Between 1970 and 1972, the anthropologist Wayne W. Zachary observed the social interactions of the 34 members of a university karate club and recorded, vertex by vertex, who interacted with whom outside of class. This dataset became known as Zachary\u0026rsquo;s Karate Club and became one of the most studied graphs in social network analysis.\nDuring the study, a conflict between the karate instructor (nicknamed \u0026ldquo;Mr. Hi\u0026rdquo;) and the club administrator (\u0026ldquo;John A.\u0026rdquo;) led to the split of the group: half of the members formed a new club around Mr. Hi, and the rest stayed with another instructor or gave up karate. The most remarkable part is that Zachary managed to predict the split using only the structure of the graph: a maximum-flow algorithm applied to the interaction network got the destination of 33 of the 34 members right, missing only one (member 9 in Zachary\u0026rsquo;s numbering, which goes from 1 to 34; in networkx, which numbers from 0, it is vertex 8), whose actual choice differed from what the network structure suggested.\nnetworkx already ships this graph, which lets us verify the Handshaking Theorem on a real network (not just on examples with 7 or 8 vertices) and use the degree to find the most influential members:\nimport networkx as nx G = nx.karate_club_graph() degrees = dict(G.degree()) degree_sum = sum(degrees.values()) num_edges = G.number_of_edges() print(f\u0026#34;Sum of degrees: {degree_sum}\u0026#34;) print(f\u0026#34;2 x |E|: {2 * num_edges}\u0026#34;) print(f\u0026#34;Theorem verified: {degree_sum == 2 * num_edges}\u0026#34;) max_degree = max(degrees.values()) hubs = [v for v, d in degrees.items() if d == max_degree] print(f\u0026#34;Highest-degree vertex(es): {hubs} (degree {max_degree})\u0026#34;) Sum of degrees: 156 2 x |E|: 156 Theorem verified: True Highest-degree vertex(es): [33] (degree 17) The theorem is confirmed even on a network with 34 vertices and 78 edges. And the maximum-degree result is no coincidence: vertex 33 is exactly the club administrator (club: \u0026quot;Officer\u0026quot;), and the runner-up, the vertex 0 with degree 16, is the instructor (club: \u0026quot;Mr. Hi\u0026quot;), the two poles around which the group split. The degree, a purely local and easy-to-compute measure, already points to the two most central figures in the network.\nThe figure below shows the complete graph, colored by the two factions that resulted from the split, with the two hubs (vertices 0 and 33) highlighted by a black outline.\nZachary\u0026rsquo;s Karate Club: factions and hubs Corollaries # Corollary 1: The sum of the degrees of all the vertices of any graph is always an even number.\nProof: By the Handshaking Theorem, \\(\\sum_{v \\in V} d(v) = 2|E|\\), which is even by definition (a multiple of 2). Therefore, the sum of the degrees of any graph is always even, which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary 2: In any graph, the number of vertices of odd degree is always even.\nProof: Split the vertices into two groups: \\(V_{\\text{even}}\\) (even degree) and \\(V_{\\text{odd}}\\) (odd degree). Then: $$\\sum_{v \\in V_{\\text{even}}} d(v) + \\sum_{v \\in V_{\\text{odd}}} d(v) = 2|E|$$ The first sum is even (a sum of even terms). Since the total \\(2|E|\\) is also even, the second sum (the sum of the odd degrees) must also be even. But a sum of odd numbers is even only when there is an even number of them. Therefore, the number of vertices of odd degree in any graph is always even, which is what we wanted to prove. \\(\\blacksquare\\)\nCorollary 3: In a \\(k\\)-regular graph with \\(n\\) vertices: $$|E| = \\frac{nk}{2}$$Proof: Let \\(G\\) be a \\(k\\)-regular graph with \\(n\\) vertices and \\(m = |E|\\). By the Handshaking Theorem: $$\\underbrace{k + k + \\cdots + k}_{n \\text{ terms}} = \\sum_{v \\in V} d(v) = 2m$$ Hence \\(nk = 2m\\), that is, \\(m = \\dfrac{nk}{2}\\). Therefore, every \\(k\\)-regular graph with \\(n\\) vertices has exactly \\(\\dfrac{nk}{2}\\) edges, which is what we wanted to prove. \\(\\blacksquare\\)\nImplication of Corollary 3 For a \\(k\\)-regular graph with \\(n\\) vertices to exist, \\(nk\\) must be even. In particular:\nIf \\(k\\) is even, any \\(n\\) works If \\(k\\) is odd, \\(n\\) must be even Applying Corollary 3 Question: Is there a 3-regular graph with 7 vertices?\nIf it existed, it would have \\(|E| = 7 \\times 3 / 2 = 10.5\\) edges, which is impossible! Therefore, there is no 3-regular graph with 7 vertices.\nDegree Sequence # Degree Sequence The degree sequence of a graph \\(G\\) with \\(n\\) vertices is the list of the degrees of all the vertices, in nondecreasing order (allowing repetitions when necessary): $$(d_1, d_2, \\ldots, d_n) \\quad \\text{with } d_1 \\leq d_2 \\leq \\cdots \\leq d_n$$ Example 7 Problem: Determine the degree sequence of the graph \\(G\\) from Example 1.\nSolution: The degrees computed were \\(d(u)=0\\), \\(d(t)=1\\), \\(d(y)=2\\), \\(d(w)=2\\), \\(d(v)=3\\), \\(d(x)=4\\), \\(d(z)=4\\). Sorting in nondecreasing order: $$(0,\\ 1,\\ 2,\\ 2,\\ 3,\\ 4,\\ 4)$$Example 7: degree sequence of G Example 8 Problem: Determine the degree sequence of the cube graph \\(G_4\\).\nSolution: All 8 vertices have degree 3, so the degree sequence is: $$(3,\\ 3,\\ 3,\\ 3,\\ 3,\\ 3,\\ 3,\\ 3)$$Example 8: degree sequence of the cube Exercises # The following exercises apply the Handshaking Theorem and its corollaries in different contexts.\nExercise 1: Number of vertices of odd degree Problem: At a party with 30 people, each person shook hands with a specific number of other people. Prove that the number of people who shook hands an odd number of times is even.\nSolution Model the party as a graph: people are vertices, handshakes are edges. By Corollary 2 of the Handshaking Theorem, the number of vertices of odd degree is always even. Therefore, the number of people who shook hands an odd number of times is even, which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 2: Pigeonhole principle for degrees Problem: Prove that in any simple graph with \\(n \\geq 2\\) vertices, there are at least two vertices with the same degree.\nSolution The possible degrees in a simple graph with \\(n\\) vertices are \\(0, 1, 2, \\ldots, n-1\\), that is, \\(n\\) possible values for \\(n\\) vertices. However, the values \\(0\\) and \\(n-1\\) cannot occur simultaneously: an isolated vertex (degree \\(0\\)) cannot coexist with a universal vertex (degree \\(n-1\\)), since the universal vertex would be adjacent to the isolated one, which is a contradiction. Hence, there are at most \\(n-1\\) distinct degree values available for \\(n\\) vertices. By the pigeonhole principle, at least two vertices must share the same degree. Therefore, every simple graph with \\(n \\geq 2\\) vertices has two vertices of the same degree, which is what we wanted to prove. \\(\\blacksquare\\)\nSummary # The following table summarizes the main concepts and results presented in this article.\nConcept Notation Description Degree of \\(v\\) \\(d(v)\\), \\(d_G(v)\\) Number of edges incident to \\(v\\) Minimum degree \\(\\delta(G)\\) Smallest degree in the graph Maximum degree \\(\\Delta(G)\\) Largest degree in the graph \\(k\\)-regular graph - Every vertex has degree \\(k\\) Handshaking Theorem \\(\\sum d(v) = 2|E|\\) Sum of degrees = twice the number of edges Corollary 1 - Sum of the degrees of any graph is even Corollary 2 - Number of vertices of odd degree is even Corollary 3 \\(|E| = \\dfrac{nk}{2}\\) Edges of a \\(k\\)-regular graph with \\(n\\) vertices Degree sequence \\((d_1, \\ldots, d_n)\\) Degrees listed in nondecreasing order Next Steps # The degree of each vertex and the degree sequence of a graph are useful tools, but they have a clear limit: two graphs can have exactly the same degree sequence and still be completely different structures, or, conversely, look different only because their vertices were labeled differently, when in fact they have the same structure. We lack tools to answer the question \u0026ldquo;are these two graphs essentially the same graph?\u0026rdquo;. This is the question that motivates graph isomorphism, the topic of the next article in the series, where we will also see how to represent graphs by adjacency and incidence matrices.\n","date":"2026-08-06","externalUrl":null,"permalink":"/2026/08/vertex-degree/","section":"Blog","summary":"The sum of the degrees of a graph is always twice the number of edges: this is the Handshaking Theorem. We will see degree, regular graphs, and the degree sequence, and we check the theorem on Zachary’s Karate Club, a real network of 34 people that split in two.","title":"Degree of a Vertex and the Handshaking Lemma","type":"posts"},{"content":" Before exploring the deep results of graph theory, we need to build a precise vocabulary. Like any area of mathematics, graph theory has specific definitions that let us communicate ideas clearly and without ambiguity.\nIn this article, we establish the formal language we will use throughout the series.\nWhy Do We Need a Formal Language? # Without a shared vocabulary, communication in graph theory would be imprecise and ambiguous. A formal language is indispensable for at least four reasons:\nModeling: real problems (computer networks, delivery routes, dependencies between tasks) can only be analyzed mathematically after they are translated into a graph with precise definitions. Communication: when two researchers say \u0026ldquo;neighborhood of \\(v\\)\u0026rdquo;, the definitions guarantee that both understand exactly the same set. Proofs: without rigorous definitions, proofs lose their validity; every step of a proof rests on the exact definition of the terms used. Implementation: the formal definitions directly guide the data structures (adjacency lists, incidence matrices) that graph algorithms use. These concepts have direct counterparts in computing: the neighborhood \\(N(v)\\) is exactly the adjacency list used in algorithms such as Breadth-First Search (BFS) and Depth-First Search (DFS); cliques are the core of community detection algorithms in social networks; and independent sets underlie the graph coloring problem, in which adjacent vertices cannot receive the same color, a restriction present in task scheduling, radio frequency allocation, and much more.\nFormal Definition of a Graph # Simple Graph A simple graph, or graph, \\(G\\) is a pair \\((V, E)\\), denoted \\(G = (V, E)\\), where \\(V\\) is a finite nonempty set of elements called vertices (also called nodes or points) and \\(E\\) is a set of unordered pairs of distinct elements of \\(V\\), called edges.\nNotation conventions Vertices = nodes = points: the three terms are synonyms. Number of vertices of \\(G\\): \\(|V(G)| = n\\). Number of edges of \\(G\\): \\(|E(G)| = m\\). When there are several graphs \\(G_1, G_2, \\ldots, G_k\\), we denote their vertex and edge sets respectively by \\((V(G_1), E(G_1))\\), \\((V(G_2), E(G_2))\\), …, \\((V(G_k), E(G_k))\\). Graphs = simple graphs: unless stated otherwise, \u0026ldquo;graph\u0026rdquo; always means simple graph in this series. Given a graph \\(G = (V, E)\\), each edge \\(e \\in E\\) will be denoted by the pair \\(e = (v, w)\\) that forms it, with \\(v, w \\in V\\). In this case, the vertices \\(v\\) and \\(w\\) are the endpoints of the edge \\(e\\). The edge \\(e\\) is said to be incident to the vertices \\(v\\) and \\(w\\).\nExample 1 Let \\(G = (V, E)\\) with: $$V = \\{a,\\ b,\\ c,\\ d,\\ e\\}$$ $$E = \\{(a,b),\\ (b,c),\\ (a,c),\\ (a,e),\\ (c,d),\\ (e,d)\\}$$In this graph:\n\\(a\\) and \\(b\\) are adjacent; \\(b\\) and \\(d\\) are not adjacent The edge \\((c,d)\\) is incident to \\(c\\) and to \\(d\\); the endpoints of \\((c,d)\\) are \\(c\\) and \\(d\\) \\(n = |V| = 5\\) and \\(m = |E| = 6\\) The figure below shows the geometric representation of \\(G\\).\nExample 1: graph G Geometric Representation # The usual way to represent a graph is geometric (in the plane): each vertex corresponds to a point in the plane and each edge \\((v, w)\\) corresponds to a line joining the points that correspond to the vertices \\(v\\) and \\(w\\).\nExample 2 Let \\(G\\) be the graph of Example 1, with \\(V = \\{a,b,c,d,e\\}\\) and \\(E = \\{(a,b),(b,c),(a,c),(a,e),(c,d),(e,d)\\}\\). This graph admits multiple valid geometric representations: the vertices can be placed at different positions in the plane (for example, in the shape of a pentagon or in any other configuration), as long as the connections are preserved. The geometric representation of a graph is not unique.\nThe two representations below correspond to the same graph \\(G\\): same vertices, same edges, different positions.\nExample 2: two representations of the same graph Multigraphs If we relax the definition of a simple graph to admit equal unordered pairs of distinct elements of \\(V\\), called parallel edges, and pairs of equal elements \\((v, v)\\), called loops, we obtain a multigraph. In this series, unless noted otherwise, we work only with simple graphs.\nExample 3 Let \\(G = (V, E)\\) with \\(V = \\{a, b, c, d\\}\\) and $$E = \\{(a,b),\\ (a,b),\\ (b,c),\\ (c,d),\\ (d,a),\\ (c,c)\\}$$ The two occurrences of \\((a,b)\\) are parallel edges; \\((c,c)\\) is a loop.\nThe multigraph below illustrates parallel edges and a loop.\nExample 3: multigraph Neighborhood and Isolation # Neighborhood Let \\(G = (V, E)\\) be a graph. The vertices \\(v\\) and \\(w \\in V\\) are said to be adjacent or neighbors if \\((v, w) \\in E\\). We call the adjacency or neighborhood of \\(v\\) in \\(G\\) the set: $$N(v) = \\{w \\in V \\mid (v, w) \\in E\\}$$ When more than one graph is at play, we write \\(N_G(v)\\) to indicate the neighborhood of \\(v\\) in the graph \\(G\\).\nExample 4 Let \\(G\\) be the graph of Example 1. We compute the neighborhood of each vertex:\n\\(N(a) = \\{b, c, e\\}\\) \\(N(b) = \\{a, c\\}\\) \\(N(c) = \\{b, a, d\\}\\) \\(N(d) = \\{c, e\\}\\) \\(N(e) = \\{a, d\\}\\) The following figure shows \\(G\\) with the vertex \\(a\\) and its edges highlighted, illustrating \\(N(a) = \\{b, c, e\\}\\).\nExample 4: neighborhood N(v) for each vertex A vertex \\(v\\) with \\(N(v) = \\emptyset\\) is called an isolated vertex, since it is not connected to any other vertex.\nExample 5 Let \\(G = (V, E)\\) with \\(V = \\{1, 2, 3, 4, 5\\}\\) and $$E = \\{(1,2),\\ (2,3),\\ (1,3),\\ (1,4),\\ (3,4)\\}$$ We compute \\(N(5) = \\emptyset\\). Hence, 5 is an isolated vertex in \\(G\\).\nVertex 5 (highlighted) has no incident edges, confirming \\(N(5) = \\emptyset\\).\nExample 5: isolated vertex A vertex \\(v\\) of \\(G\\) is said to be universal when $$N(v) = V - \\{v\\}$$ that is, \\(v\\) is adjacent to all the other vertices of the graph.\nExample 6 Let \\(G = (V, E)\\) with \\(V = \\{1, 2, 3, 4, 5\\}\\) and $$E = \\{(1,2),\\ (1,3),\\ (1,4),\\ (1,5),\\ (2,3),\\ (3,4),\\ (4,5)\\}$$ We compute \\(N(1) = \\{2, 3, 4, 5\\} = V - \\{1\\}\\). Hence, 1 is a universal vertex in \\(G\\).\nVertex 1 (highlighted) is connected to all the others: \\(N(1) = V - \\{1\\}\\).\nExample 6: universal vertex Isolated and universal do not coexist In a graph with at least two vertices we cannot have an isolated vertex and a universal vertex at the same time: the universal vertex would be a neighbor of the isolated one, which by definition has no neighbors. The exception is the single-vertex graph, in which \\(N(v) = \\emptyset = V - \\{v\\}\\) and the vertex is both.\nThe flowchart below summarizes how to classify any vertex \\(v\\) as isolated, universal, or neither.\n\u0026#x2197; flowchart TD A[\"Vertex v in G\"] --\u003e B{\"N(v) = ∅ ?\"} B -- \"Yes\" --\u003e C[\"Isolated\nno neighbors\"] B -- \"No\" --\u003e D{\"N(v) = V − {v} ?\"} D -- \"Yes\" --\u003e E[\"Universal\nadjacent to all\"] D -- \"No\" --\u003e F[\"Neither isolated\nnor universal\"] The Complement Graph # Complement Graph The complement of \\(G\\), denoted \\(\\bar{G}\\), is the graph with the same vertex set \\(V(G)\\), such that two distinct vertices are adjacent in \\(\\bar{G}\\) if and only if they are not adjacent in \\(G\\): $$V(\\bar{G}) = V(G)$$ $$E(\\bar{G}) = \\{(v, w) \\mid v, w \\in V(G)\\ \\text{and}\\ (v, w) \\notin E(G)\\}$$ Example 7 Let \\(G\\) be the graph of Example 1: \\(V = \\{a,b,c,d,e\\}\\) and \\(E(G) = \\{(a,b),(b,c),(a,c),(a,e),(c,d),(e,d)\\}\\).\nThe complete graph on 5 vertices would have \\(\\binom{5}{2} = 10\\) edges. Since \\(m = 6\\), the complement \\(\\bar{G}\\) has \\(10 - 6 = 4\\) edges, exactly the pairs that do not appear in \\(G\\): $$E(\\bar{G}) = \\{(a,d),\\ (b,d),\\ (b,e),\\ (c,e)\\}$$The two graphs below illustrate \\(G\\) and \\(\\bar{G}\\): note that the edges present in one are absent from the other.\nExample 7: G and its complement Subgraphs # One of the most important notions is that of a subgraph: parts of a graph obtained by keeping a subset of its vertices and edges.\nSubgraph Let \\(G\\) and \\(H\\) be graphs. \\(H\\) is said to be a subgraph of \\(G\\), denoted \\(H \\subseteq G\\), if:\n\\(V(H) \\subseteq V(G)\\) \\(E(H) \\subseteq E(G)\\) Example 8 Let \\(G\\) and \\(H\\) be the following graphs:\n\\(V(G) = \\{a, b, c, d, e\\}\\),\n\\(E(G) = \\{(a,b),\\ (b,c),\\ (a,c),\\ (c,d),\\ (e,d),\\ (a,e)\\}\\)\n\\(V(H) = \\{a, b, c, d\\}\\), \\(E(H) = \\{(a,b),\\ (b,c)\\}\\)\nSince \\(V(H) \\subseteq V(G)\\) and \\(E(H) \\subseteq E(G)\\), we conclude that \\(H\\) is a subgraph of \\(G\\).\nThe figure shows \\(G\\) on the left and \\(H\\) on the right. Note that \\(H\\) has only 2 of the 6 edges of \\(G\\), and the vertex \\(d\\) is isolated in \\(H\\).\nExample 8: subgraph H of G Induced Subgraph # Induced Subgraph Let \\(G\\) be a graph and \\(A \\subseteq V(G)\\), that is, a subset of the vertices of \\(G\\). The subgraph of \\(G\\) induced by \\(A\\), denoted \\(G[A]\\), is defined by: $$V(G[A]) = A$$ $$E(G[A]) = \\{(x, y) \\in E(G) \\mid x \\in A\\ \\text{and}\\ y \\in A\\}$$ \\(G[A]\\) inherits all the edges of \\(G\\) whose two endpoints belong to \\(A\\).\nExample 9 Considering the graph \\(G\\) of Example 8, let \\(A = \\{a, b, c, d\\}\\) (\\(A \\subset V(G)\\)). The subgraph \\(F = G[A]\\) induced by \\(A\\) has:\n\\(V(F) = \\{a, b, c, d\\}\\)\n\\(E(F) = \\{(a,b),\\ (b,c),\\ (a,c),\\ (c,d)\\}\\)\nObserve that \\((a,c)\\) and \\((c,d)\\) belong to \\(E(F)\\) because both endpoints of each of these edges belong to \\(A\\) and they exist in \\(E(G)\\).\nIn the figure, the vertices of \\(A\\) are highlighted in \\(G\\) (left); on the right, \\(F = G[A]\\) with all 4 inherited edges.\nExample 9: induced subgraph F = G[A] Subgraph ≠ Induced Subgraph The subgraph \\(H\\) of Example 8 and the subgraph \\(F\\) of Example 9 have the same vertex set \\(\\{a,b,c,d\\}\\), but they are not equal:\n\\(E(H) = \\{(a,b),\\ (b,c)\\}\\): \\(H\\) is not the subgraph of \\(G\\) induced by the set \\(\\{a,b,c,d\\}\\), because the edges \\((a,c)\\) and \\((c,d)\\) exist in \\(G\\) with both endpoints in \\(\\{a,b,c,d\\}\\), but were omitted from \\(H\\). \\(E(F) = \\{(a,b),\\ (b,c),\\ (a,c),\\ (c,d)\\}\\): \\(F\\) is an induced subgraph because it inherited all the edges of \\(G\\) among the vertices of \\(A\\). The comparison below places \\(G\\), \\(H\\), and \\(F\\) side by side: \\(H\\) and \\(F\\) have the same \\(V\\), but \\(H\\) omits edges that \\(F\\) necessarily inherits.\nComparison: subgraph vs induced subgraph Spanning Subgraph # Spanning Subgraph Let \\(G\\) and \\(H\\) be graphs. \\(H\\) is said to be a spanning subgraph of \\(G\\) if \\(H\\) is a subgraph of \\(G\\) and \\(V(H) = V(G)\\), that is, \\(H\\) contains all the vertices of \\(G\\), but possibly fewer edges.\nExample 10 Considering the graph \\(G\\) of Example 8, let \\(H\\) be given by:\n\\(V(H) = V(G) = \\{a, b, c, d, e\\}\\), \\(E(H) = \\{(a,b),\\ (a,c),\\ (c,d)\\}\\)\nSince \\(V(H) = V(G)\\) and \\(E(H) \\subseteq E(G)\\), \\(H\\) is a spanning subgraph of \\(G\\).\nThe spanning subgraph \\(H\\) keeps all 5 vertices of \\(G\\), but preserves only 3 of the 6 edges, and the vertex \\(e\\) is isolated.\nExample 10: spanning subgraph The following diagram summarizes the relationships among the types of subgraph studied and their special cases.\n\u0026#x2197; flowchart TD A[\"Subgraph H ⊆ G\nV(H) ⊆ V(G) and E(H) ⊆ E(G)\"] --\u003e B[\"Induced Subgraph G[A]\ninherits all edges of G\nwhose two endpoints are in A\"] A --\u003e C[\"Spanning Subgraph\nV(H) = V(G)\npossibly fewer edges\"] B --\u003e D[\"Clique A\nG[A] is complete (= Kk)\"] B --\u003e E[\"Independent Set S\nG[S] is null (= Ns)\"] Special Graphs: Complete and Null # Complete Graph \\(K_n\\) A graph \\(G\\) is said to be complete if all of its pairs of distinct vertices are adjacent. In other words, all of its vertices are universal. A complete graph with \\(n\\) vertices is denoted by \\(K_n\\).\nNumber of edges: \\(|E(K_n)| = \\binom{n}{2} = \\dfrac{n(n-1)}{2}\\)\nExample 11 \\(K_1\\): 1 vertex, 0 edges \\(K_2\\): 2 vertices, 1 edge \\(K_3\\): 3 vertices, 3 edges \\(K_4\\): 4 vertices, 6 edges \\(K_5\\): 5 vertices, 10 edges The first five complete graphs are shown below.\nExample 11: complete graphs K1 to K5 Clique # Clique Let \\(G\\) be a graph and \\(A \\subseteq V(G)\\) a subset of vertices. \\(A\\) is a clique of \\(G\\) if \\(G[A]\\) is a complete graph: every pair of vertices in \\(A\\) is adjacent.\nExample 12 Let \\(G\\) be a graph with \\(V = \\{a,b,c,d,e,f,g,h\\}\\) and edges: $$(a,b),\\ (a,c),\\ (a,d),\\ (b,c),\\ (b,d),\\ (c,d),\\ (c,e),\\ (e,f),\\ (c,f),\\ (c,h),\\ (h,g),\\ (g,f)$$The following subsets are cliques of \\(G\\):\n\\(A_1 = \\{a, b, c, d\\}\\): clique of size 4 (\\(G[A_1] = K_4\\)) \\(A_2 = \\{c, e, f\\}\\): clique of size 3 (\\(G[A_2] = K_3\\)) \\(A_3 = \\{g, h\\}\\): clique of size 2 Other \\(A_i\\) are also possible: any subset of \\(A_1\\) with 2 or 3 vertices (for example \\(\\{a,b,c\\}\\), \\(\\{a,c,d\\}\\)) forms a clique. A clique is maximal when it is not contained in any larger clique, and maximum when no clique of the graph has more vertices than it does. \\(A_1\\), \\(A_2\\), and \\(A_3\\) are maximal, as are \\(\\{c,h\\}\\) and \\(\\{f,g\\}\\), but only \\(A_1\\), with 4 vertices, is maximum. Finding a maximum clique in a large graph is a hard problem: no efficient algorithm is known for the general case (see the entry Clique problem).\nThe figure below shows \\(G\\) with the vertices colored by group: \\(A_1\\) in blue, \\(A_2\\) in orange, \\(A_3\\) in green; the vertex \\(c\\) (shared by \\(A_1\\) and \\(A_2\\)) is highlighted.\nExample 12: cliques in G Null Graph # The natural counterpart of \\(K_n\\) is the null graph: instead of all pairs being adjacent, no pair is adjacent.\nNull Graph \\(N_n\\) A graph \\(G\\) is said to be null (or completely independent) if none of its pairs of distinct vertices are adjacent. In other words, all of its vertices are isolated. A null graph with \\(n\\) vertices is denoted by \\(N_n\\).\nExample 13 \\(N_1\\): 1 vertex, 0 edges \\(N_2\\): 2 vertices, 0 edges \\(N_3\\): 3 vertices, 0 edges \\(N_4\\): 4 vertices, 0 edges \\(N_5\\): 5 vertices, 0 edges The first five null graphs are shown below.\nExample 13: null graphs N1 to N5 Independent Set # Independent Set Let \\(G\\) be a graph and \\(S \\subseteq V(G)\\) a subset of vertices. \\(S\\) is an independent set (or stable set) of \\(G\\) if \\(G[S]\\) is a null graph: no pair of vertices in \\(S\\) is adjacent.\nExample 14 Let \\(G\\) be the graph of Example 12. The following subsets are independent sets of \\(G\\):\n\\(S_1 = \\{a, e, g\\}\\) \\(S_2 = \\{e, h\\}\\) \\(S_3 = \\{d, h, f\\}\\) The figure shows \\(G\\) with the independent set \\(S_1 = \\{a, e, g\\}\\) highlighted: no pair of these vertices shares an edge.\nExample 14: independent set S1 Clique / independent set duality If \\(A\\) is a clique in \\(G\\), then \\(A\\) is an independent set in the complement \\(\\bar{G}\\), and vice versa. This follows directly from the definitions: the edges present in \\(G[A]\\) are absent from \\(\\bar{G}[A]\\), turning a complete subgraph into a null subgraph.\nExercises # The following exercises consolidate the concepts presented: neighborhood, complement, subgraphs, and independence.\nExercise 1: Neighborhood and special vertices Statement:\nGiven the graph \\(G = (V, E)\\) with \\(V = \\{1, 2, 3, 4, 5\\}\\) and $$E = \\{(1,2),\\ (1,3),\\ (2,4),\\ (3,4),\\ (4,5)\\}$$a) Compute \\(N(v)\\) for each vertex \\(v \\in V\\).\nb) Is there any isolated vertex? And any universal vertex?\nSolution a) Going through the edges for each vertex:\n\\(N(1) = \\{2, 3\\}\\) \\(N(2) = \\{1, 4\\}\\) \\(N(3) = \\{1, 4\\}\\) \\(N(4) = \\{2, 3, 5\\}\\) \\(N(5) = \\{4\\}\\) b) No vertex has \\(N(v) = \\emptyset\\), so there is no isolated vertex. No vertex has \\(N(v) = V - \\{v\\}\\) (for example, \\(N(1) = \\{2,3\\} \\neq \\{2,3,4,5\\}\\)), so there is no universal vertex.\nThe graph \\(G\\) with all neighborhoods annotated:\nExercise 1: graph G and neighborhoods Exercise 2: Complement graph Statement:\nLet \\(G\\) be the graph of Exercise 1. Determine \\(\\bar{G}\\).\nSolution \\(K_5\\) has \\(\\binom{5}{2} = 10\\) edges. Since \\(|E(G)| = 5\\), the complement has \\(10 - 5 = 5\\) edges, which are the pairs that do not appear in \\(G\\): $$E(\\bar{G}) = \\{(1,4),\\ (1,5),\\ (2,3),\\ (2,5),\\ (3,5)\\}$$\\(G\\) and \\(\\bar{G}\\) side by side:\nExercise 2: G and complement Exercise 3: Induced subgraph and independent set Statement:\nLet \\(G\\) be the graph of Exercise 1 and \\(A = \\{1, 2, 4\\}\\).\na) Determine the induced subgraph \\(G[A]\\).\nb) Is \\(S = \\{1, 5\\}\\) an independent set of \\(G\\)? Justify.\nSolution a) \\(V(G[A]) = \\{1, 2, 4\\}\\). The edges of \\(G\\) with both endpoints in \\(A\\) are \\((1,2)\\) and \\((2,4)\\). Therefore: $$E(G[A]) = \\{(1,2),\\ (2,4)\\}$$b) Yes. The only pair in \\(S\\) is \\(\\{1, 5\\}\\), and \\((1,5) \\notin E(G)\\). Hence \\(G[S]\\) is a null graph and \\(S\\) is an independent set.\nOn the left: \\(G\\) with \\(A = \\{1,2,4\\}\\) in orange and the edges of \\(G[A]\\) highlighted. On the right: \\(S = \\{1,5\\}\\) in green, with no edge between them.\nExercise 3: induced subgraph and independent set Summary # Concept Symbol/Notation Description Simple graph \\(G = (V, E)\\) Vertices + edges (unordered pairs) Edge \\(e = (v, w)\\) Unordered pair; \\(v\\) and \\(w\\) are the endpoints Endpoints of an edge \\(v, w\\) The two vertices that form the edge \\((v, w)\\) Incidence n/a \\(e = (v, w)\\) is incident to the vertices \\(v\\) and \\(w\\) Number of vertices \\(n\\) \\(n = \\lvert V(G) \\rvert\\) Number of edges \\(m\\) \\(m = \\lvert E(G) \\rvert\\) Neighborhood \\(N(v)\\), \\(N_G(v)\\) Set of neighbors of \\(v\\) in \\(G\\) Isolated vertex n/a \\(N(v) = \\emptyset\\) Universal vertex n/a \\(N(v) = V - \\{v\\}\\) Complement \\(\\bar{G}\\) Same \\(V\\), complementary edges Subgraph \\(H \\subseteq G\\) \\(V(H) \\subseteq V(G)\\) and \\(E(H) \\subseteq E(G)\\) Induced subgraph \\(G[A]\\) Vertices of \\(A\\) + all inherited edges Spanning subgraph n/a \\(V(H) = V(G)\\), possibly fewer edges Complete graph \\(K_n\\) Every pair adjacent; \\(\\binom{n}{2}\\) edges Null graph \\(N_n\\) No edges; all vertices isolated Clique n/a \\(A \\subseteq V(G)\\) with \\(G[A]\\) complete Independent set n/a \\(S \\subseteq V(G)\\) with \\(G[S]\\) null Next Steps # With the vocabulary established (vertices, edges, neighborhood, subgraphs, and special graphs), we have the basis to start measuring local properties of each vertex. What is still missing is quantifying how many edges are incident to each vertex: this measure, called the degree, is the central object of the next article, where we will also prove the Handshaking Theorem, which relates the sum of the degrees to the number of edges of the graph.\n","date":"2026-08-04","externalUrl":null,"permalink":"/2026/08/graph-definitions/","section":"Blog","summary":"The formal vocabulary of graph theory: vertices, edges, neighborhood, isolated and universal vertices, complement, induced and spanning subgraphs, complete and null graphs, cliques and independent sets, with 14 worked examples and exercises.","title":"Definitions and Notation in Graph Theory","type":"posts"},{"content":" Imagine you are on vacation in the city of Königsberg, in 18th-century Prussia. The Pregel River divides the city into four distinct regions, connected by seven bridges. The challenge occupying the minds of the locals is seemingly simple: would it be possible to take a walk through the city crossing each of the seven bridges exactly once?\nThis question, which looks like a trivial pastime, gave rise to one of the most applied fields of modern mathematics: graph theory.\nThe Königsberg Bridge Problem # In 1736, the Swiss mathematician Leonhard Euler received this challenge and realized that the geographic question (the exact positions of the bridges, the distances between the islands) was irrelevant. All that mattered was the structure of the connections: which regions were linked by bridges and how many bridges connected them.\nThe map below shows the actual layout of the seven bridges over the Pregel River, the starting point of Euler\u0026rsquo;s reasoning.\nHistorical map of the seven bridges of Königsberg over the Pregel River, in 18th-century Prussia. Source: Wikimedia Commons Euler abstracted the problem: he represented each region as a point and each bridge as a line connecting two points. With this representation, the question became: is it possible to traverse every line of this diagram exactly once, in a single continuous route?\nEuler\u0026rsquo;s model Labeling each region with a letter, we count its bridges:\nA (Altstadt, north bank): 3 bridges B (Kneiphof, central island): 5 bridges C (Lomse, east island): 3 bridges D (Vorstadt, south bank): 3 bridges Result: all four \u0026ldquo;points\u0026rdquo; have an odd number of connections.\nThe historical names Königsberg was the German name of the city; since 1946 it has been called Kaliningrad and belongs to Russia.\nAltstadt (lit. \u0026ldquo;old town\u0026rdquo;): north bank of the Pregel River Kneiphof: central island of the Pregel, now called Kant Island, where the cathedral in which Immanuel Kant is buried stands Lomse: island east of Kneiphof, separated by an arm of the river Vorstadt (lit. \u0026ldquo;suburb\u0026rdquo;): south bank of the Pregel River The graph below represents this abstraction, with vertices in place of the regions and edges in place of the bridges. The two parallel edges between A and B (and between B and D) represent the two bridges that existed between those regions.\nKönigsberg graph: A=Altstadt, B=Kneiphof, C=Lomse, D=Vorstadt. The double edges represent two bridges between the same regions; the total is seven. Euler\u0026rsquo;s answer was no, and he showed it with a simple argument. Notice that, each time the walker enters a region by one bridge, they must leave by another. Every passage through an intermediate region therefore consumes a pair of bridges. This means that any region that is neither the starting point nor the end point must have an even number of bridges. Only the start and the end of the route can have an odd number. In Königsberg, however, all four points have an odd number of bridges, which would require four start or end points at the same time, something impossible in a single continuous route.\nMore precisely: a route that traverses each edge exactly once is called an Eulerian trail. If it returns to the starting point, it is an Eulerian circuit, and for one to exist, all vertices must have even degree. If it starts and ends at distinct points, exactly two vertices may have odd degree. In Königsberg, with four vertices of odd degree, neither condition is satisfied. In the article Paths, Cycles, and Connectivity, we will formalize the distinction between walk, trail, and path; in Eulerian and Hamiltonian Graphs, we will see the rigorous proof of this result and its generalizations.\nThe bridges of Königsberg today Two of the seven bridges were destroyed in the bombings of World War II, and two others were demolished afterward; of the three that remained, only two date from Euler\u0026rsquo;s time. Today there are five bridges in the same places as in the original problem. With this configuration, two vertices have degree 2 and the other two have degree 3, and today it is possible to cross all the existing bridges exactly once, starting on one island and ending on the other. The structure changed and the answer changed with it, which illustrates that in graph theory the answer depends on the structure, not on the geography.\nThis work, published in the paper Solutio problematis ad geometriam situs pertinentis (1736), is considered the birth of graph theory (see the entry Seven Bridges of Königsberg).\nWhat Is a Graph? # The central idea that Euler developed is very simple:\nGraph (intuitive definition) A graph is a set of vertices (points, nodes) and a set of edges (lines, connections) that link pairs of vertices.\nThere is no geometry, no coordinates, no distances: only the structure of who is connected to whom.\nThis abstraction seems like an impoverishment, but it allows the same theory to apply to completely different domains.\nAnother Example: the Four Color Theorem # The Bridges of Königsberg are not the only classic problem that the language of graphs clarifies. Consider any geographic map: a political map, a neighborhood map, a board divided into regions. We want to color the regions so that two countries (or neighborhoods) that share a border never have the same color. How many colors are needed?\nThis question was raised in 1852 by the mathematician Francis Guthrie, while he was coloring a map of the counties of England, and it remained open for 124 years. Intuitively, it seems that four colors should be enough for any planar map, but proving it turned out to be very hard.\nThe connection with graph theory is immediate: just represent each region as a vertex and draw an edge between two vertices whenever the corresponding regions share a border. The map coloring problem then becomes a graph coloring problem: assign colors to the vertices so that adjacent vertices receive different colors.\nThe following figure illustrates this abstraction with a map of four regions (A, B, C and D) that are mutually adjacent, each colored with a distinct color, and the corresponding graph, where all the edges reflect the adjacencies of the map.\nFour mutually adjacent regions require four distinct colors (left). The corresponding graph is K₄, the complete graph on four vertices, where every pair of regions are neighbors. In this specific example, the four regions are all mutually adjacent: A borders B, C, and D; B borders A, C, and D; and so on. The resulting graph is K₄, the complete graph on four vertices, in which every pair of vertices is connected by an edge. To color it, we necessarily need four colors.\nFour Color Theorem Every planar map can be colored with at most four colors so that neighboring regions (those that share a border of positive length) receive distinct colors.\nWhat makes this theorem remarkable is its proof. In 1976, Kenneth Appel and Wolfgang Haken published a proof that reduced the problem to a finite (but enormous) set of cases to check, and used a computer to verify all of them. It was the first major mathematical proof to depend on a computer, and it sparked a philosophical debate about what it means to \u0026ldquo;prove\u0026rdquo; something in mathematics that continues to this day (see the entry Four color theorem).\nTwo geometric facts lie behind the statement:\nIn any planar map, it is impossible to have five mutually adjacent regions (which would require the graph K₅, which is not planar). The graph K₄ we saw above is planar (it can be drawn in the plane without crossings) and shows that four colors are sometimes necessary. These two facts, however, do not prove that four colors suffice: a map may need more colors than its largest group of mutually neighboring regions suggests. A small example is a central region surrounded by a ring of five regions. In this map there are no four mutually neighboring regions, and yet four colors are needed: the ring, with an odd number of regions, already uses up three, and the central region, which neighbors all of them, needs a fourth. It is precisely this easy intuition, that everything comes down to how many regions touch at the same time, that no one managed to turn into a proof during the 124 years the problem remained open.\nThe theory of planar graphs and Kuratowski\u0026rsquo;s planarity criterion will be discussed in detail in the article Planar Graphs, which also returns to coloring: the Four Color Theorem stated for graphs, a proof that six colors always suffice, and applications in scheduling.\nWhy Are Graphs So Useful? # Graph theory is a universal language for modeling relationships between objects. The key is to realize that, in many real problems, what matters is not the objects themselves, but the connections between them.\nComputer Science # Computer networks: routers are vertices, cables are edges. Finding the shortest path between two routers is a graph problem. Operating systems: dependency graphs between processes help detect deadlocks. Compilers: a program\u0026rsquo;s control-flow analysis uses directed graphs. Databases: the entity-relationship model is essentially a graph. Search algorithms: BFS (breadth-first search) and DFS (depth-first search) are algorithms on graphs. Engineering and Logistics # Electrical networks: identifying circuits, analyzing energy flow. Route planning: the Traveling Salesman Problem asks for the Hamiltonian cycle of least cost in a weighted graph. Construction projects: precedence graphs determine the order of activities (PERT/CPM method, Program Evaluation and Review Technique / Critical Path Method). Natural and Social Sciences # Chemistry: molecules are graphs where atoms are vertices and chemical bonds are edges. Structural isomers have the same atoms linked in different ways (their graphs are not isomorphic), and recognizing whether two structural formulas describe the same molecule is a graph isomorphism problem. Biology: protein-protein interaction networks, food webs (who eats whom), phylogenetic trees. Sociology: social networks, where people are vertices and friendships are edges. The concept of \u0026ldquo;six degrees of separation\u0026rdquo; is a statement about the diameter of a graph. Epidemiology: modeling the spread of diseases in contact networks. The Internet and Social Networks # The World Wide Web itself is an enormous graph, but with a difference from the Königsberg graph and the map coloring graph, both undirected: it is a directed graph. Each page is a vertex and each hyperlink is an edge with a direction. If Page A points to Page B, that does not imply that B points to A. Direction matters: the number of pages that point to a vertex (its in-degree) is a direct measure of how referenced (and potentially relevant) that page is.\nThe following graph illustrates a mini-web with five pages. The arrows indicate the direction of the hyperlinks, and the size of each node is proportional to the PageRank computed by the algorithm, which anticipates a result that is not immediately obvious.\nA fragment of the Web as a directed graph. Each arrow is a hyperlink; the size of each node is proportional to its PageRank (value shown below the label). B receives links from A, C, and E (the highest in-degree), but D and A have higher PageRank, thanks to the feedback cycle D→A. In this example, Page \\(B\\) receives links from three distinct pages (\\(A\\), \\(C\\), and \\(E\\)), and is the most referenced in terms of in-degree. Page \\(E\\), on the other hand, receives no incoming links. There is also a cycle: following \\(A \\to C \\to D \\to A\\), the navigator returns to the starting point. These properties (in-degree, absence of incoming links, and cycles) are exactly what PageRank models, with results that sometimes surprise.\nPageRank is the algorithm developed by Larry Page and Sergey Brin at Stanford University in 1996, which became the basis of Google\u0026rsquo;s search engine. The central idea is the random surfer model: imagine an internet user who, at each step, randomly chooses one of the links on the current page and follows it. The PageRank of a page is the probability that this surfer is on it after a very large number of steps, that is, a measure of how frequently the page is visited in the long run.\n$$PR(A) = \\frac{1-d}{N} + d \\sum_{v \\to A} \\frac{PR(v)}{L(v)}$$where \\(d \\approx 0.85\\) is the damping factor (the probability of continuing to click instead of \u0026ldquo;starting over\u0026rdquo; on a random page), \\(N\\) is the total number of pages, and \\(L(v)\\) is the number of links leaving \\(v\\). Pages that receive many links from pages that are themselves highly ranked accumulate high PageRank, an iterative process that converges to a stable distribution.\nThe animation below shows this convergence process: the circles represent pages, the size of each circle indicates the PageRank computed in the current iteration, and the arrows represent the hyperlinks.\nIterative convergence of PageRank on a network of pages. The size of each node reflects its computed importance: heavily referenced pages grow over the iterations. Source: Wikimedia Commons The origin of the name The name \u0026ldquo;PageRank\u0026rdquo; is a double play on words: it honors Larry Page, one of its creators, and it describes what the algorithm does, which is to rank pages (pages). The original 1998 paper, The Anatomy of a Large-Scale Hypertextual Web Search Engine, is a widely cited reference in the field.\nComputing the PageRank of our mini-web. To apply the algorithm to the graph in the diagram above, we need to build the transition matrix \\(M\\), a \\(5 \\times 5\\) matrix where the entry \\(M_{ij}\\) represents the fraction of the PageRank of page \\(j\\) that is transferred to page \\(i\\) at each step. The construction rule is straightforward: if page \\(j\\) has \\(k\\) outgoing links and one of them points to \\(i\\), then \\(M_{ij} = \\frac{1}{k}\\); otherwise, \\(M_{ij} = 0\\). Since each page distributes all of its weight among its outgoing links, each column of \\(M\\) sums to 1.\nNumbering the nodes as \\(A=0\\), \\(B=1\\), \\(C=2\\), \\(D=3\\), \\(E=4\\) and reading the edges of the graph:\nColumn \\(A\\) (\\(A\\) has 2 outgoing links: to \\(B\\) and \\(C\\)): \\(M_{BA} = 0.5\\), \\(M_{CA} = 0.5\\), all others = 0. Column \\(B\\) (\\(B\\) has 1 outgoing link: to \\(D\\)): \\(M_{DB} = 1.0\\). Column \\(C\\) (\\(C\\) has 2 outgoing links: to \\(B\\) and \\(D\\)): \\(M_{BC} = 0.5\\), \\(M_{DC} = 0.5\\). Column \\(D\\) (\\(D\\) has 1 outgoing link: to \\(A\\)): \\(M_{AD} = 1.0\\). Column \\(E\\) (\\(E\\) has 1 outgoing link: to \\(B\\)): \\(M_{BE} = 1.0\\). The row of \\(E\\) is entirely zero: no one points to \\(E\\), so \\(E\\) only receives the teleportation \u0026ldquo;floor\u0026rdquo; \\(\\frac{1-d}{N}\\).\n$$M = \\begin{pmatrix} 0.0 \u0026 0.0 \u0026 0.0 \u0026 1.0 \u0026 0.0 \\\\ 0.5 \u0026 0.0 \u0026 0.5 \u0026 0.0 \u0026 1.0 \\\\ 0.5 \u0026 0.0 \u0026 0.0 \u0026 0.0 \u0026 0.0 \\\\ 0.0 \u0026 1.0 \u0026 0.5 \u0026 0.0 \u0026 0.0 \\\\ 0.0 \u0026 0.0 \u0026 0.0 \u0026 0.0 \u0026 0.0 \\end{pmatrix} \\quad \\text{(rows: } A, B, C, D, E \\text{; columns: } A, B, C, D, E\\text{)}$$Connection with the previous formula. Recall the PageRank formula presented above:\n$$PR(A) = \\frac{1-d}{N} + d \\sum_{v \\to A} \\frac{PR(v)}{L(v)}$$The matrix \\(M\\) was built precisely to represent this summation. The entry \\(M_{ij}\\) captures the term \\(\\frac{PR(j)}{L(j)}\\): when \\(j\\) links to \\(i\\), \\(M_{ij} \\cdot PR(j) = \\frac{PR(j)}{L(j)}\\); otherwise it is zero. The product \\(M \\cdot \\mathbf{PR}\\) in row \\(i\\) adds up all these contributions, which is precisely \\(\\sum_{v \\to i} \\frac{PR(v)}{L(v)}\\). Therefore, the scalar formula above is equivalent to the vector equation\n$$\\mathbf{PR} = \\frac{1-d}{N} \\mathbf{1} + d M \\mathbf{PR}$$where \\(\\mathbf{1}\\) is the vector of ones and \\(\\mathbf{PR}\\) is the vector of ranks. The following algorithm solves this equation by the power iteration method: it starts with the uniform distribution \\(\\mathbf{w} = \\tfrac{1}{N}\\mathbf{1}\\) and repeatedly applies \\(\\mathbf{v} = d M \\mathbf{w} + \\tfrac{1-d}{N}\\mathbf{1}\\) until convergence, which corresponds exactly to the line v = M_hat @ w + (1 - d) / N in the code below.\nWith the matrix in hand, we can apply the algorithm to our mini-web. The resulting vector contains the normalized PageRanks (sum = 1) of \\(A\\), \\(B\\), \\(C\\), \\(D\\), and \\(E\\), respectively:\nimport numpy as np def pagerank(M, d=0.85): N = M.shape[1] w = np.ones(N) / N # initial uniform distribution M_hat = d * M v = M_hat @ w + (1 - d) / N while np.linalg.norm(w - v) \u0026gt;= 1e-10: w = v v = M_hat @ w + (1 - d) / N return v # M[i][j] = fraction of the links of j that point to i M = np.array([ [0.0, 0.0, 0.0, 1.0, 0.0], # A receives 100% from D [0.5, 0.0, 0.5, 0.0, 1.0], # B receives 50% from A, 50% from C, and 100% from E [0.5, 0.0, 0.0, 0.0, 0.0], # C receives 50% from A [0.0, 1.0, 0.5, 0.0, 0.0], # D receives 100% from B and 50% from C [0.0, 0.0, 0.0, 0.0, 0.0], # E receives from no one ]) pages = [\u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, \u0026#34;E\u0026#34;] ranks = pagerank(M) for page, rank in sorted(zip(pages, ranks), key=lambda x: -x[1]): print(f\u0026#34;Page {page}: {rank:.4f}\u0026#34;) Page D: 0.2974 Page A: 0.2828 Page B: 0.2395 Page C: 0.1502 Page E: 0.0300 The result is counterintuitive: \\(B\\) has the highest in-degree (it receives links from \\(A\\), \\(C\\), and \\(E\\)), but it is not the node with the highest PageRank. \\(D\\) and \\(A\\) come out ahead. Why? \\(B\\) passes 100% of its rank to \\(D\\) (since its only outgoing link points to \\(D\\)). \\(D\\), in turn, passes 100% to \\(A\\). And \\(A\\) redistributes to \\(B\\) and \\(C\\), which feed \\(D\\) again. The cycle \\(D \\to A \\to C \\to D\\) and \\(D \\to A \\to B \\to D\\) creates a feedback loop that accumulates rank in \\(D\\) and \\(A\\) over the iterations. \\(B\\) receives a lot, but also gives away a lot, keeping nothing for itself.\n\\(E\\) confirms the opposite case: with no incoming links, its PageRank is just the teleportation \u0026ldquo;floor,\u0026rdquo; \\(\\frac{1 - 0.85}{5} = 0.03\\), regardless of where it points.\nSocial networks are also modeled as graphs, but with an important distinction regarding the direction of the edges. On Facebook, friendship is mutual by definition: if Alice is a friend of Bob, Bob is a friend of Alice, and the graph is undirected. On Instagram or Twitter/X, following someone does not imply being followed back: the graph is directed, with the edge A→B indicating \u0026ldquo;A follows B\u0026rdquo;.\nThe following graph models a small social network with six users and friendship relations, and is therefore undirected.\nA social network with six users. Alice (orange) and David have degree 3, the most connected nodes. Alice, Bob, and David form a triangle: all three are friends with one another. Carol, Eve, and Frank connect to the core but are not directly connected to everyone else. Alice and David have degree 3 (the highest local connectivity in the network) and form, together with Bob, a triangle: the three are mutual friends. Carol is connected only to Alice and Eve; Frank, only to David and Eve. It is already possible to see that some vertices concentrate more connections than others and that the position of a vertex in the network influences its ability to \u0026ldquo;reach\u0026rdquo; the others in a few hops. This notion of centrality (how strategically positioned a vertex is in the network) is one of the central metrics in social network analysis.\nA famous result in this context is the six degrees of separation hypothesis: the idea that any two people in the world are at most six \u0026ldquo;hops\u0026rdquo; apart in the global social network. In 2011, Facebook researchers analyzed the platform\u0026rsquo;s social graph (then with more than 700 million users) and found an average distance of 4.74 hops between two users (3.74 intermediaries), below the six of the hypothesis, in a real-scale network.\nThe Beauty of Abstraction # Graph theory operates at the pure level of structure. A benzene molecule, a social network with ten thousand users, and the São Paulo subway map can all be modeled as graphs and studied with the same mathematical tools.\nEuler did not know, in 1736, that he was founding a theory that would be central to computing, telecommunications, bioinformatics, and social network analysis in the following centuries. He just wanted to solve a puzzle about bridges.\nIn the next article, we will formalize the language of graph theory: definitions, notation, and the first fundamental examples.\nThe Articles in the Series # Throughout this series, we will build the foundations of graph theory systematically. We will start with the basic definitions and move on to deep results and their applications:\nGraphs: A Language for Modeling Connections (this article) Definitions and Notation in Graph Theory: the formal language (vertices, edges, neighborhood, subgraphs, complete graphs) Degree of a Vertex and the Handshaking Lemma: degree, regular graphs, and the theorem that relates degrees and edges Isomorphism and Matrix Representation: when two graphs are \u0026ldquo;the same\u0026rdquo; and how to represent them computationally Paths, Cycles, and Connectivity: how to navigate a graph; bipartite graphs and their characterization Trees: connected, acyclic graphs, widely used in computing Eulerian and Hamiltonian Graphs: special routes that traverse all edges or all vertices Planar Graphs: when a graph can be drawn without edge crossings Directed Graphs: when the direction of the connections matters ","date":"2026-07-30","externalUrl":null,"permalink":"/2026/07/graphs-introduction/","section":"Blog","summary":"Can you cross the seven bridges of Königsberg passing over each one exactly once? Euler showed you can’t, and in doing so created graph theory. We will see what a graph is, the Four Color Theorem, and the computation of PageRank on a mini-web of five pages, in Python.","title":"Graphs: A Language for Modeling Connections","type":"posts"},{"content":"","date":"2026-07-29","externalUrl":null,"permalink":"/tags/networks/","section":"tags","summary":"","title":"networks","type":"tags"},{"content":"","date":"2026-07-29","externalUrl":null,"permalink":"/tags/redes/","section":"tags","summary":"","title":"redes","type":"tags"},{"content":" In 1736, Leonhard Euler proposed a solution to a puzzle about bridges in a Prussian city. In doing so, he accidentally founded one of the most widely applied branches of mathematics: graph theory.\nWhy does graph theory matter? # In computing and mathematics, modeling relationships between objects shows up everywhere:\nComputer networks: packet routing and failure analysis depend on graph algorithms, and distance and connectivity are the subject of Paths, Cycles, and Connectivity Search engines: Google\u0026rsquo;s PageRank ranks pages by computing centrality in a graph of the web, and the calculation appears step by step in Graphs: A Language for Modeling Connections Navigation: map and GPS (Global Positioning System) navigation apps compute shortest paths with accelerated variants of Dijkstra\u0026rsquo;s algorithm (breadth-first search solves the unweighted case), both presented in Paths, Cycles, and Connectivity Social networks: analyzing communities, influencers, and the spread of information are graph problems. Zachary\u0026rsquo;s Karate Club, a real network that split in two, appears in Degree of a Vertex and in Paths, Cycles, and Connectivity Compilers and dependencies: analyzing modules and detecting cycles in software projects use directed graphs. Dependency cycles appear in Paths, Cycles, and Connectivity, and topological sorting, in Directed Graphs Bioinformatics: genome assembly and protein interaction networks are modeled as graphs. Genome assembly by Eulerian trails appears in Eulerian and Hamiltonian Graphs Graph theory provides the language and the algorithms to handle these problems with mathematical rigor.\nThis series covers the foundations of graph theory in a systematic and accessible way, starting from the basic definitions and advancing to results such as Euler\u0026rsquo;s Theorem and Kuratowski\u0026rsquo;s Theorem.\nWhat You Will Learn # Over the 9 articles in this series, we will build the foundation of graph theory:\nGraphs: A Language for Modeling Connections # The story of the Bridges of Königsberg problem, the fundamental idea behind graphs, and why this abstraction serves so many problems. We will see how the same mathematical language models computer networks, chemical molecules, social networks, and much more.\n→ Graphs: A Language for Modeling Connections\nDefinitions and Notation in Graph Theory # The formal language: simple graph \\(G = (V, E)\\), neighborhood, isolated and universal vertex, complement of a graph, subgraphs (induced and spanning), complete graphs \\(K_n\\), null graphs \\(N_n\\), cliques, and independent sets.\n→ Definitions and Notation in Graph Theory\nDegree of a Vertex and the Handshaking Lemma # The degree \\(d(v)\\) of a vertex, minimum degree \\(\\delta(G)\\) and maximum degree \\(\\Delta(G)\\), regular graphs, and the Handshaking Theorem: \\(\\sum d(v) = 2|E|\\). Corollaries about parity and degree sequences.\n→ Degree of a Vertex and the Handshaking Lemma\nIsomorphism and Matrix Representation # When are two graphs structurally identical? The definition of isomorphism, necessary conditions for checking it, and how to represent graphs computationally: adjacency matrix and incidence matrix.\n→ Isomorphism and Matrix Representation\nPaths, Cycles, and Connectivity # How to navigate a graph: the distinctions between walk, trail, path, and cycle. Connectivity, connected components, distance, diameter, center, and how to compute distances with breadth-first search and Dijkstra\u0026rsquo;s algorithm. Bipartite graphs and their characterization by odd cycles.\n→ Paths, Cycles, and Connectivity\nTrees: The Most Elegant Structure in Graphs # Trees as connected acyclic graphs. The fundamental theorem \\(m = n-1\\), uniqueness of paths, leaves, center (1 or 2 vertices), spanning trees, and the ubiquity of rooted trees in computing.\n→ Trees: The Most Elegant Structure in Graphs\nEulerian and Hamiltonian Graphs # Special tours: the Eulerian circuit traverses every edge, the Hamiltonian cycle visits every vertex. Euler\u0026rsquo;s Theorem, the Dirac and Ore theorems for Hamiltonicity, and the famous Traveling Salesman Problem.\n→ Eulerian and Hamiltonian Graphs\nPlanar Graphs: When Crossing Is Unavoidable # When is it possible to draw a graph without crossing edges? Euler\u0026rsquo;s Formula \\(n - m + f = 2\\), corollaries proving that \\(K_5\\) and \\(K_{3,3}\\) are not planar, graph subdivisions, Kuratowski\u0026rsquo;s Theorem, and the coloring of planar graphs, with the Four Color Theorem.\n→ Planar Graphs: When Crossing Is Unavoidable\nDirected Graphs: When Direction Matters # Digraphs to model asymmetric relationships. In-degree and out-degree, sources and sinks, strong/unilateral/weak connectivity, nonsymmetric adjacency matrices, and applications in networks and dependency analysis, using DAGs (Directed Acyclic Graphs) and topological sorting.\n→ Directed Graphs: When Direction Matters\nThe articles were designed to be read in order, and the diagram below shows why. Besides the sequence, the dashed arrows mark results from one article that come back as a tool in a later article:\n\u0026#x2197; flowchart TD A1[\"Graphs: a language\nfor modeling connections\"] --\u003e A2[\"Definitions\nand notation\"] A2 --\u003e A3[\"Degree and the\nHandshaking Lemma\"] A3 --\u003e A4[\"Isomorphism\nand matrices\"] A4 --\u003e A5[\"Paths, cycles\nand connectivity\"] A5 --\u003e A6[\"Trees\"] A6 --\u003e A7[\"Eulerian and\nHamiltonian\"] A7 --\u003e A8[\"Planar graphs\"] A8 --\u003e A9[\"Directed graphs\"] A3 -.-\u003e|\"Handshaking\nin Christofides\"| A7 A5 -.-\u003e|\"bipartite:\nK3,3 has no triangles\"| A8 A6 -.-\u003e|\"spanning tree\nin Euler's Formula\"| A8 A3 -.-\u003e|\"in-degree\nand out-degree\"| A9 Who Is This Series For? # This series was written for those who are familiar with basic mathematics (high school level) and want to understand the mathematical foundations that underpin computer science and network analysis.\nNo programming experience is needed. The concepts are presented with concrete examples, worked exercises, and connections to real applications.\nPrerequisites # The concepts in this series connect naturally with other discrete mathematics topics covered here on the site:\nSet theory: the language of sets is everywhere. See the series The Language of Sets Combinatorics: counting paths, cycles, and structures in graphs. See the series The Art of Counting Mathematical induction: used in the proofs of the fundamental theorems. See the series From Base Case to Infinity If you are not yet familiar with these topics, you can follow the other series before studying graph theory.\nWhat Comes Before and After # This series is part of a larger sequence on Discrete Mathematics:\nBefore: The Language of Sets, From Base Case to Infinity, and The Art of Counting, which cover sets, mathematical induction, and combinatorics Now: graph theory, with connections, structure, and algorithms Each series can be read independently, but the progression was planned to build intuition gradually.\nQuick Reference Table # The topics and main results of each article in this series:\nArticle Topic Main result / concept 1 Graphs: A Language Modeling relationships as graphs 2 Definitions and Notation \\(G = (V, E)\\), cliques, subgraphs 3 Degree and Handshaking \\(\\sum d(v) = 2\\lvert E \\rvert\\) 4 Isomorphism and Matrices Adjacency matrix, isomorphism 5 Paths and Connectivity Connected components, distance, breadth-first search, bipartite graphs 6 Trees \\(m = n - 1\\), spanning trees 7 Eulerian and Hamiltonian Euler\u0026rsquo;s Theorem, Dirac\u0026rsquo;s Theorem 8 Planar Graphs \\(n - m + f = 2\\), Kuratowski\u0026rsquo;s Theorem, Four Color Theorem 9 Directed Graphs Digraphs, DAGs, strong connectivity Ready to start? Let\u0026rsquo;s begin where it all started: the Bridges of Königsberg.\nEnjoy the reading!\n","date":"2026-07-29","externalUrl":null,"permalink":"/2026/07/mathematics-of-connections/","section":"Blog","summary":"In 1736, Euler solved a puzzle about the bridges of a Prussian city and, without meaning to, founded graph theory. This series of 9 articles goes from the basic definitions to Kuratowski’s Theorem, through paths, trees, planarity, and directed graphs.","title":"The Mathematics of Connections: Graph Theory from Scratch","type":"posts"},{"content":"Uma jornada pela Combinatória: dos Princípios Aditivo e Multiplicativo ao Binômio de Newton, passando por permutações, arranjos e combinações — com e sem repetição. O roteiro completo para dominar a arte de contar possibilidades.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-07-28","externalUrl":null,"permalink":"/series/a-arte-de-contar/","section":"Séries","summary":"","title":"A Arte de Contar","type":"series"},{"content":"","date":"2026-07-28","externalUrl":null,"permalink":"/tags/combinatoria/","section":"tags","summary":"","title":"combinatória","type":"tags"},{"content":"","date":"2026-07-28","externalUrl":null,"permalink":"/tags/combinatorics/","section":"tags","summary":"","title":"combinatorics","type":"tags"},{"content":"A journey through Combinatorics: from the Addition and Multiplication Principles to the Binomial Theorem, by way of permutations, arrangements and combinations, with and without repetition. The complete roadmap to mastering the art of counting possibilities.\nThe articles are listed below, in the order they were written.\n","date":"2026-07-28","externalUrl":null,"permalink":"/series/the-art-of-counting/","section":"Series","summary":"","title":"The Art of Counting","type":"series"},{"content":" What is the coefficient of \\(x^7\\) in the expansion of \\((x + 2)^{10}\\)? Computing it directly (multiplying ten factors and tracking every term) takes dozens of operations. The binomial theorem reduces this to one line, and the coefficients that show up are exactly the \\(\\binom{n}{r}\\) of Pascal\u0026rsquo;s triangle that we saw in the previous article.\nWhat did Newton do, anyway? The expansion of \\((a+b)^n\\) for integer \\(n\\) is not Newton\u0026rsquo;s: it was known centuries before him, with the same coefficients as the triangle. Isaac Newton\u0026rsquo;s contribution, between 1664 and 1665, was to extend the formula to any real exponent, including fractional and negative ones. In that case the sum does not end: \\((1+x)^{1/2} = 1 + \\frac{1}{2}x - \\frac{1}{8}x^2 + \\frac{1}{16}x^3 - \\cdots\\) is an infinite sum, the binomial series, valid for \\(|x| \u0026lt; 1\\). This article deals with the integer-exponent case, which is the one combinatorics uses.\nWhy the binomial theorem? # The binomial theorem shows up in many places in computing and mathematics:\nProbability: the binomial distribution (a model of \\(n\\) independent experiments with success probability \\(p\\)) writes \\((p + q)^n\\) as a sum of terms \\(\\binom{n}{k} p^k q^{n-k}\\), where each term is the probability of exactly \\(k\\) successes. Numerical approximations: for \\(|x| \\ll 1\\), the expansion \\((1+x)^n \\approx 1 + nx + \\binom{n}{2}x^2 + \\cdots\\) underlies numerical methods and Taylor series in calculus. Computer algebra: SymPy implements expand((a+b)**n) with multinomial coefficients, the generalization of the binomial theorem, and expand has already appeared on the site in the section on creating polynomials with SymPy. Knowing the formula lets you predict and debug symbolic results. Combinatorial identities: substituting specific values for \\(a\\) and \\(b\\) yields identities about binomial coefficients, including the Row Theorem (\\(\\sum \\binom{n}{k} = 2^n\\)), presented in the previous article, whose most direct proof comes from here. Polynomial kernels in machine learning: the number of terms in \\((x_1 + x_2 + \\cdots + x_d)^n\\) is directly tied to combinations with repetition, and the binomial is the case \\(d = 2\\). The Map of the Theorem # The diagram below shows how the concepts in this article connect: the binomial theorem generates the general term, which lets us extract specific coefficients, and special cases produce identities about rows of Pascal\u0026rsquo;s triangle.\n\u0026#x2197; flowchart TD TH[\"Binomial theorem\n(a+b)ⁿ = Σ C(n,r) aⁿ⁻ʳ bʳ\"] --\u003e TG[\"General term\nT_{r+1} = C(n,r) aⁿ⁻ʳ bʳ\"] TH --\u003e SUB[\"Special cases\nby substitution\"] TG --\u003e COEF[\"Extract the coefficient\nof xᵏ: solve n−r=k\"] SUB --\u003e I1[\"a=b=1\n∑ C(n,k) = 2ⁿ\"] SUB --\u003e I2[\"a=1, b=−1\n∑ (−1)ʳ C(n,r) = 0\"] I1 --\u003e PAL[\"Recovers the Row\nTheorem (previous article)\"] The Binomial Theorem # Before stating the theorem, look at the expansions for small \\(n\\): the coefficients in each row are exactly the values of the corresponding row of Pascal\u0026rsquo;s triangle.\n\\(n\\) Expansion 0 \\(1\\) 1 \\(a + b\\) 2 \\(a^2 + 2ab + b^2\\) 3 \\(a^3 + 3a^2b + 3ab^2 + b^3\\) 4 \\(a^4 + 4a^3b + 6a^2b^2 + 4ab^3 + b^4\\) The figure below makes the correspondence for \\(n = 4\\): each number in the row drops onto the term it is the coefficient of, and below each term appears the corresponding binomial coefficient.\nRow 4 of Pascal\u0026rsquo;s triangle gives the coefficients of (a + b)⁴ Binomial Theorem For all \\(a, b \\in \\mathbb{R}\\) and every integer \\(n \\geq 0\\): $$(a + b)^n = \\sum_{r=0}^{n} \\binom{n}{r} a^{n-r} b^r$$ Proof by Mathematical Induction # Mathematical induction (a proof technique that treats the base case and the inductive step separately) is the natural tool here, since the binomial theorem claims something for every integer \\(n \\geq 0\\). Since the base is \\(n = 0\\), we use the generalized version of the principle, with \\(n_0 = 0\\) (in the convention of this collection of series, \\(0 \\notin \\mathbb{N}\\)).\nBase case (\\(n=0\\)): \\((a+b)^0 = 1 = \\binom{0}{0} a^0 b^0\\). ✓\nInductive hypothesis (IH): suppose that, for some integer \\(k \\geq 0\\), \\((a+b)^k = \\sum_{r=0}^{k}\\binom{k}{r}a^{k-r}b^r\\) holds.\nInductive step (\\(n = k+1\\)): multiplying both sides of the IH by \\((a+b)\\):\n$$ (a+b)^{k+1} = (a+b)\\sum_{r=0}^{k}\\binom{k}{r}a^{k-r}b^r = \\sum_{r=0}^{k}\\binom{k}{r}a^{k+1-r}b^r + \\sum_{r=0}^{k}\\binom{k}{r}a^{k-r}b^{r+1} $$Reindexing the second sum with \\(r \\to r-1\\):\n$$= \\sum_{r=0}^{k}\\binom{k}{r}a^{k+1-r}b^r + \\sum_{r=1}^{k+1}\\binom{k}{r-1}a^{k+1-r}b^r$$The extreme terms (\\(r=0\\) and \\(r=k+1\\)) contribute \\(a^{k+1}\\) and \\(b^{k+1}\\); for the intermediate ones, Pascal\u0026rsquo;s rule gives \\(\\binom{k}{r-1} + \\binom{k}{r} = \\binom{k+1}{r}\\):\n$$= \\sum_{r=0}^{k+1}\\binom{k+1}{r}a^{k+1-r}b^r$$Therefore, \\((a+b)^n = \\sum_{r=0}^{n} \\binom{n}{r} a^{n-r} b^r\\) for every integer \\(n \\geq 0\\), which is what we wanted to prove. \\(\\blacksquare\\)\nGeneral Term # The expansion of \\((a+b)^n\\) has \\(n+1\\) terms. To identify a specific term without expanding everything, we use the general term formula:\nGeneral Term The \\((r+1)\\)-th term of the expansion of \\((a+b)^n\\) is: $$T_{r+1} = \\binom{n}{r} a^{n-r} b^r, \\qquad r = 0, 1, \\ldots, n$$ Identities from Special Cases # Substituting specific values for \\(a\\) and \\(b\\) in the binomial theorem gives identities about binomial coefficients.\nIdentity 1: Sum of the triangle\u0026rsquo;s rows (\\(a = b = 1\\)) $$\\sum_{r=0}^n \\binom{n}{r} = 2^n$$ This result coincides with the Row Theorem from the previous article: the binomial theorem gives its most direct proof.\nIdentity 2: Alternating signs (\\(a = 1, b = -1\\)) For \\(n \\geq 1\\) (with \\(n = 0\\), the sum has a single term and equals 1): $$\\sum_{r=0}^n (-1)^r\\binom{n}{r} = 0$$ Corollary: adding the even and odd indices separately and using Identity 1: $$\\binom{n}{0} + \\binom{n}{2} + \\cdots = \\binom{n}{1} + \\binom{n}{3} + \\cdots = 2^{n-1}$$ Worked Examples # The diagram below summarizes the procedure for applying the general term: it branches depending on whether the goal is to fully expand the binomial or to isolate the coefficient of a specific term.\n\u0026#x2197; flowchart TD A[\"Write (a+b)ⁿ:\nidentify a, b and n\"] --\u003e B[\"General term:\nT_{r+1} = C(n,r) aⁿ⁻ʳ bʳ\"] B --\u003e C{\"What is the goal?\"} C --\u003e|\"Full expansion\"| D[\"Compute r = 0, 1, …, n\nand add the n+1 terms\"] C --\u003e|\"Coefficient of aᵏbʲ\"| E[\"Solve n − r = k\n(or r = j)\"] E --\u003e F{\"r ∈ {0, …, n}?\"} F --\u003e|\"Yes\"| G[\"Compute T_{r+1} = C(n,r) aⁿ⁻ʳ bʳ\"] F --\u003e|\"No\"| H[\"Coefficient = 0\n(the term does not exist)\"] Full expansion of \\((x+3)^4\\) # Problem: Expand \\((x+3)^4\\) using the binomial theorem.\nSolution: The general term is \\(T_{r+1} = \\binom{4}{r} x^{4-r} 3^r\\). Computing each term:\n\\(r\\) \\(\\binom{4}{r}\\) \\(x^{4-r}\\) \\(3^r\\) Term 0 1 \\(x^4\\) 1 \\(x^4\\) 1 4 \\(x^3\\) 3 \\(12x^3\\) 2 6 \\(x^2\\) 9 \\(54x^2\\) 3 4 \\(x\\) 27 \\(108x\\) 4 1 1 81 \\(81\\) $$(x+3)^4 = \\mathbf{x^4 + 12x^3 + 54x^2 + 108x + 81}$$ Coefficient of \\(x^2\\) in \\(\\left(x^3 - \\dfrac{1}{x^2}\\right)^9\\) # Problem: Find the coefficient of \\(x^2\\) in the expansion of \\(\\left(x^3 - \\dfrac{1}{x^2}\\right)^9\\).\nSolution: The general term is:\n$$T_{r+1} = \\binom{9}{r}(x^3)^{9-r}\\left(-\\frac{1}{x^2}\\right)^r = \\binom{9}{r}(-1)^r x^{27-3r-2r} = \\binom{9}{r}(-1)^r x^{27-5r}$$For the term to contain \\(x^2\\), we need \\(27 - 5r = 2\\), that is, \\(r = 5\\):\n$$T_6 = \\binom{9}{5}(-1)^5 x^2 = 126 \\cdot (-1) \\cdot x^2 = -126x^2$$The coefficient of \\(x^2\\) is \\(\\mathbf{-126}\\).\nPython: extracting coefficients with math.comb The general term formula translates directly into code. To recover the coefficient of \\(x^7\\) in \\((x + 2)^{10}\\) (the opening question) we just solve \\(10 - r = 7 \\Rightarrow r = 3\\) and compute:\nfrom math import comb def binomial_coef(b, n, exp_a): \u0026#34;\u0026#34;\u0026#34;Coefficient of a^exp_a in the expansion of (a + b)^n.\u0026#34;\u0026#34;\u0026#34; r = n - exp_a # exponent of b in the term return comb(n, r) * (b ** r) # Coefficient of x^7 in (x + 2)^10 print(binomial_coef(b=2, n=10, exp_a=7)) # 960 # Check: the sum of row 10 must be 2^10 = 1024 print(sum(binomial_coef(1, 10, k) for k in range(11))) # 1024 The second block checks Identity 1: substituting \\(a = b = 1\\), the sum of all the coefficients of \\((1+1)^{10}\\) must give \\(2^{10} = 1{,}024\\).\nSummary Table # The table below gathers all the results proved in this article for quick reference.\nResult Formula Binomial theorem \\(\\displaystyle(a+b)^n = \\sum_{r=0}^{n} \\binom{n}{r} a^{n-r} b^r\\) General term \\(T_{r+1} = \\binom{n}{r} a^{n-r} b^r\\) Identity 1 (\\(a=b=1\\)) \\(\\displaystyle\\sum_{r=0}^n \\binom{n}{r} = 2^n\\) Identity 2 (\\(a=1, b=-1\\), \\(n \\geq 1\\)) \\(\\displaystyle\\sum_{r=0}^n (-1)^r\\binom{n}{r} = 0\\) Corollary (parities, \\(n \\geq 1\\)) \\(\\displaystyle\\sum_{r \\text{ even}} \\binom{n}{r} = \\sum_{r \\text{ odd}} \\binom{n}{r} = 2^{n-1}\\) Exercises # The following exercises apply the binomial theorem to expansions, extraction of specific terms, and proofs of identities. They are ordered from simplest to most complex.\nExercise 1: Expansions Expand: (a) \\((2a - 3b)^3\\) (b) \\((2y + 3x)^4\\) (c) \\(\\left(\\dfrac{1}{y} - y\\right)^6\\)\nSolution (a) \\(T_{r+1} = \\binom{3}{r}(2a)^{3-r}(-3b)^r\\): $$8a^3 - 36a^2b + 54ab^2 - 27b^3$$(b) \\(T_{r+1} = \\binom{4}{r}(2y)^{4-r}(3x)^r\\): $$16y^4 + 96y^3x + 216y^2x^2 + 216yx^3 + 81x^4$$(c) \\(T_{r+1} = \\binom{6}{r}(y^{-1})^{6-r}(-y)^r = \\binom{6}{r}(-1)^r y^{2r-6}\\): $$y^{-6} - 6y^{-4} + 15y^{-2} - 20 + 15y^2 - 6y^4 + y^6$$ Exercise 2: Sixth term Compute the 6th term of: (a) \\(\\left(1 - \\dfrac{1}{b}\\right)^7\\) (b) \\(\\left(3x^2y - \\dfrac{1}{3}\\right)^9\\)\nSolution The 6th term uses \\(r = 5\\):\n(a) \\(T_6 = \\binom{7}{5}(1)^2\\left(-\\dfrac{1}{b}\\right)^5 = -\\dfrac{21}{b^5}\\)\n(b) \\(T_6 = \\binom{9}{5}(3x^2y)^4\\left(-\\dfrac{1}{3}\\right)^5 = 126 \\cdot 81x^8y^4 \\cdot \\left(-\\dfrac{1}{243}\\right) = \\mathbf{-42x^8y^4}\\)\nExercise 3: Sum of the coefficients Compute the sum of the coefficients of all the terms of \\(\\left(x^3 - \\dfrac{1}{2x}\\right)^{12}\\).\nSolution The sum of the coefficients is the value of the expression at \\(x = 1\\): $$\\left(1 - \\frac{1}{2}\\right)^{12} = \\left(\\frac{1}{2}\\right)^{12} = \\mathbf{\\frac{1}{4{,}096}}$$ Exercise 4: Terms independent of \\(x\\) Find the term independent of \\(x\\) in: (a) \\(\\left(x^2 + \\dfrac{1}{x^2}\\right)^6\\) (b) \\(\\left(x^2 + \\dfrac{1}{x}\\right)^9\\)\nSolution (a) \\(T_{r+1} = \\binom{6}{r}x^{12-4r}\\). Independent: \\(12 - 4r = 0 \\Rightarrow r = 3\\). Term: \\(\\binom{6}{3} = \\mathbf{20}\\)\n(b) \\(T_{r+1} = \\binom{9}{r}x^{18-3r}\\). Independent: \\(18 - 3r = 0 \\Rightarrow r = 6\\). Term: \\(\\binom{9}{6} = \\mathbf{84}\\)\nExercise 5: Why does \\((x + x^{-1})^{2n+1}\\) have no independent term? Proof \\(T_{r+1} = \\binom{2n+1}{r} x^{2n+1-2r}\\). To be independent of \\(x\\), we would need \\(2n+1-2r = 0 \\Rightarrow 2r = 2n+1\\). This is impossible: the left side is even and the right side is odd. Therefore, \\((x + x^{-1})^{2n+1}\\) has no term independent of \\(x\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 6: Proof: \\(\\left(1 + \\dfrac{1}{n}\\right)^n \u0026gt; 2\\) for \\(n \\geq 2\\) Proof By the binomial theorem with \\(a = 1\\) and \\(b = \\frac{1}{n}\\): $$\\left(1 + \\frac{1}{n}\\right)^n = \\sum_{k=0}^n \\binom{n}{k}\\frac{1}{n^k}$$ The terms \\(k=0\\) and \\(k=1\\) already add up to \\(1 + 1 = 2\\). For \\(n \\geq 2\\), there are additional strictly positive terms. Therefore, \\(\\left(1+\\frac{1}{n}\\right)^n \u0026gt; 2\\) for \\(n \\geq 2\\), which is what we wanted to prove. \\(\\blacksquare\\)\nThe same expansion is one of the classic routes to the number \\(e\\): as \\(n\\) grows, \\(\\left(1+\\frac{1}{n}\\right)^n\\) increases without ever exceeding 3 and converges to \\(e = 2.71828\\ldots\\) (for \\(n = 1000\\), it is already 2.7169…).\nNext Steps # With the binomial theorem we close the series \u0026ldquo;The Art of Counting\u0026rdquo;. We started from the sum and product rules, built permutations, arrangements and combinations (with and without repetition), studied the binomial coefficients and Pascal\u0026rsquo;s triangle, and arrived at the theorem that unites algebra and combinatorics. The map of all the Discrete Mathematics content on the site is in \u0026ldquo;Discrete Mathematics: Introduction and Roadmap\u0026rdquo;.\nCombinatorics answers questions of the type \u0026ldquo;how many?\u0026rdquo;, but many problems in computing also ask \u0026ldquo;how are they connected?\u0026rdquo;. To answer that, graph theory comes in, the mathematics of networks, paths and connections. The next article, The Mathematics of Connections: Graph Theory from Scratch, introduces graphs from scratch: from the Bridges of Königsberg to modern search algorithms.\nSeries Summary # The table below gathers all the counting techniques in the series, with the formula and the situation in which to use each one.\nTechnique Formula When to use Sum rule \\(|A| + |B|\\) Exclusive choices (\u0026ldquo;or\u0026rdquo;) Product rule \\(|A| \\cdot |B|\\) Independent stages (\u0026ldquo;and\u0026rdquo;) Permutation \\(n!\\) Order everything Circular permutation \\((n-1)!\\) Order around a circle Permutation with repetition \\(\\dfrac{n!}{n_1!\\cdots n_k!}\\) Order with identical items Arrangement \\(\\dfrac{n!}{(n-r)!}\\) Order \\(r\\) out of \\(n\\) Arrangement with repetition \\(n^r\\) Sequence with repetition Combination \\(\\dbinom{n}{r}\\) Select \\(r\\) out of \\(n\\) Combination with repetition \\(\\dbinom{n+r-1}{r}\\) Select with repetition Binomial theorem \\(\\displaystyle\\sum_{r=0}^n\\binom{n}{r}a^{n-r}b^r\\) Expand \\((a+b)^n\\) ","date":"2026-07-28","externalUrl":null,"permalink":"/2026/07/binomial-theorem/","section":"Blog","summary":"What is the coefficient of a term in the expansion of a binomial raised to the tenth power? The binomial theorem answers in one line, with the coefficients from Pascal’s triangle. We will see the theorem and its proof by induction, the general term formula, and the identities that come from special cases.","title":"The Binomial Theorem: Expanding (a+b)^n","type":"posts"},{"content":" How many subsets does a set with 10 elements have? The answer, \\(1{,}024\\), does not need a direct count: it emerges from the sum of row \\(n = 10\\) of Pascal\u0026rsquo;s triangle. The binomial coefficient \\(\\binom{n}{r}\\) satisfies identities that reinforce each other and make complex calculations surprisingly direct.\nWhy binomial coefficients? # The identities of binomial coefficients reappear in central areas of computing and mathematics:\nProbability and statistics: the binomial distribution (a model of \\(n\\) independent trials with success probability \\(p\\)) has \\(P(X = k) = \\binom{n}{k} p^k (1-p)^{n-k}\\). A good part of the analysis of A/B tests, confidence intervals, and Bayesian classification models goes through this formula. Coding theory: the number of binary words of length \\(n\\) with exactly \\(k\\) bits equal to 1 (Hamming weight \\(k\\), that is, the number of 1 bits in the word) is \\(\\binom{n}{k}\\). The Row Theorem, \\(\\sum_k \\binom{n}{k} = 2^n\\), says that the sum of all weight classes gives the total number of binary words. Algorithm analysis: exhaustive search algorithms that evaluate all subsets of size \\(k\\) have complexity \\(\\Theta\\left(\\binom{n}{k}\\right)\\). The symmetry \\(\\binom{n}{k} = \\binom{n}{n-k}\\) shows that evaluating subsets of size \\(k\\) costs the same as evaluating their complements of size \\(n-k\\). Polynomial expansion: the coefficients of \\((1 + x)^n\\) are exactly \\(\\binom{n}{0}, \\binom{n}{1}, \\ldots, \\binom{n}{n}\\). In machine learning, the number of polynomial features of degree \\(\\leq d\\) generated from \\(n\\) input variables (features) is \\(\\binom{n+d}{d}\\), directly tied to combination with repetition. Recurrences and dynamic programming: Pascal\u0026rsquo;s rule \\(\\binom{n}{r} = \\binom{n-1}{r-1} + \\binom{n-1}{r}\\) is the basis for memoizing the triangle, the same pattern of overlapping subproblems that makes dynamic programming (a technique of storing partial results to reuse them) efficient. The map of identities # The diagram below shows how the identities in this article relate to each other: Pascal\u0026rsquo;s rule generates the triangle, and the triangle reveals the three sum theorems.\n\u0026#x2197; flowchart TD DEF[\"C(n,r): definition\"] --\u003e SIM[\"Symmetry\nC(n,r) = C(n,n-r)\"] DEF --\u003e FRONT[\"Boundary\nC(n,0) = C(n,n) = 1\"] DEF --\u003e PASCAL[\"Pascal's rule\nC(n,r) + C(n,r+1) = C(n+1,r+1)\"] PASCAL --\u003e TRI[\"Pascal's triangle\"] TRI --\u003e LINHA[\"Row Theorem\n∑ C(n,k) = 2ⁿ\"] TRI --\u003e COL[\"Column Theorem\n∑ C(k,r) = C(n+1,r+1)\"] TRI --\u003e DIAG[\"Diagonal Theorem\n∑ C(n+k,k) = C(n+r+1,r)\"] Binomial Coefficient # Definition $$\\binom{n}{r} = C(n, r) = \\frac{n!}{r! (n-r)!}, \\qquad 0 \\leq r \\leq n$$ Fundamental Identities # Symmetry # Identity 1: Symmetry $$\\binom{n}{r} = \\binom{n}{n-r}$$ Combinatorial proof: choosing the \\(r\\) elements that go in is equivalent to choosing the \\(n-r\\) that stay out, because each subset of \\(r\\) elements corresponds to exactly one subset of \\(n-r\\) elements, its complement. Therefore, \\(\\binom{n}{r} = \\binom{n}{n-r}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nBoundary and Secondary Identities # Identities 2 and 3 $$\\binom{n}{0} = \\binom{n}{n} = 1 \\qquad \\binom{n}{1} = \\binom{n}{n-1} = n$$ Pascal\u0026rsquo;s Rule (Stifel) # In Brazil, the rule is also commonly named after Michael Stifel, who published part of the triangle in 1544, in the Arithmetica integra.\nIdentity 4: Pascal\u0026rsquo;s Rule $$\\binom{n}{r} + \\binom{n}{r+1} = \\binom{n+1}{r+1}$$Algebraic proof: starting from the definition and using the common denominator \\((r+1)!(n-r)!\\):\n$$\\binom{n}{r} + \\binom{n}{r+1} = \\frac{n!}{r!(n-r)!} + \\frac{n!}{(r+1)!(n-r-1)!}$$$$= \\frac{n! (r+1)}{(r+1)!(n-r)!} + \\frac{n! (n-r)}{(r+1)!(n-r)!} = \\frac{n! [(r+1)+(n-r)]}{(r+1)!(n-r)!}$$$$= \\frac{n! (n+1)}{(r+1)!(n-r)!} = \\frac{(n+1)!}{(r+1)!(n-r)!} = \\binom{n+1}{r+1}$$Therefore, \\(\\binom{n}{r} + \\binom{n}{r+1} = \\binom{n+1}{r+1}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nCombinatorial proof: from \\(n+1\\) objects, in how many ways can we choose \\(r+1\\)? Fix the object \\(x\\): either it is included (we choose \\(r\\) of the \\(n\\) remaining: \\(\\binom{n}{r}\\)) or it is not (we choose \\(r+1\\) of the \\(n\\) remaining: \\(\\binom{n}{r+1}\\)). The two cases do not overlap and cover all choices, so they add up. Therefore, \\(\\binom{n}{r} + \\binom{n}{r+1} = \\binom{n+1}{r+1}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nThe figure below illustrates the rule with the concrete example \\(C(4,2) = C(3,1) + C(3,2) = 3 + 3 = 6\\):\nPascal\u0026rsquo;s rule: each element is the sum of the two above it Pascal\u0026rsquo;s Triangle # The triangle before Pascal The triangle is much older than Pascal and was discovered several times, independently:\nIndia: Pingala\u0026rsquo;s Chandaḥśāstra (3rd or 2nd century BC) already counted the combinations of long and short syllables in verses; around the year 975, Halayudha, commenting on Pingala, described the triangle explicitly, under the name Meru-prastāra. Persia: al-Karaji (953-1029) described the triangle in a book now lost, and Omar Khayyam (1048-1131) used it to extract roots. In Iran, it is Khayyam\u0026rsquo;s triangle. China: Jia Xian (1010-1070) already knew it, and Yang Hui (1238-1298) presented it in the 13th century. There, it is Yang Hui\u0026rsquo;s triangle. Europe: Petrus Apianus published the complete triangle in 1527, Stifel part of it in 1544, and Tartaglia six rows in 1556. In Italy, it is Tartaglia\u0026rsquo;s triangle. Blaise Pascal gathered the known properties and used them in probability problems in the Traité du triangle arithmétique, published after his death, in 1665, the same treatise that is often cited as the first explicit formulation of mathematical induction. The name \u0026ldquo;Pascal\u0026rsquo;s triangle\u0026rdquo; came later, with Montmort (1708) and de Moivre (1730). More at Pascal\u0026rsquo;s triangle.\nPascal\u0026rsquo;s rule lets us build the triangle iteratively: each element is the sum of the two above it in the previous row, with 1 at the ends of each row. Each position of the triangle corresponds to a binomial coefficient: the position in row \\(n\\) and column \\(r\\) contains exactly \\(C(n, r)\\). The figure below shows the triangle written in this notation:\nPascal\u0026rsquo;s triangle: position (n,r) contains C(n,r) Replacing each \\(C(n,r)\\) with its numerical value, we get the triangle that usually appears in books:\nPascal\u0026rsquo;s triangle: row n, column r contains C(n,r) In row \\(n\\), column \\(r\\) contains \\(\\binom{n}{r}\\).\nRow Theorem # Row Theorem $$\\sum_{k=0}^{n} \\binom{n}{k} = 2^n$$Proof: In the binomial theorem \\((a+b)^n\\) (the topic of the next article), substitute \\(a = b = 1\\). Therefore, \\(\\sum_{k=0}^n \\binom{n}{k} = 2^n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nInterpretation: the total number of subsets of a set with \\(n\\) elements is \\(2^n\\), and the sum of the coefficients in a row of the triangle confirms this directly. For \\(n = 10\\), this gives \\(2^{10} = 1{,}024\\), the answer to the opening question.\nThe figure below highlights row \\(n = 5\\) in the triangle, showing that \\(1 + 5 + 10 + 10 + 5 + 1 = 32 = 2^5\\):\nRow Theorem: the sum of each row is a power of 2 Column Theorem # Column Theorem (Absorption) $$\\sum_{k=r}^{n} \\binom{k}{r} = \\binom{r}{r} + \\binom{r+1}{r} + \\cdots + \\binom{n}{r} = \\binom{n+1}{r+1}$$Summing the \\(r\\)-th column of the triangle down to row \\(n\\) gives \\(\\binom{n+1}{r+1}\\), the element immediately below and to the right of the last one summed. The figure below shows the triangle in a grid and highlights column \\(r = 2\\): the values \\(1, 3, 6, 10, 15\\) add up to \\(35 = C(7,3)\\).\nColumn Theorem: the sum of column r down to row n is C(n+1,r+1) Diagonal Theorem # Diagonal Theorem $$\\sum_{k=0}^{r} \\binom{n+k}{k} = \\binom{n+r+1}{r}$$We add up elements along an ascending diagonal of the triangle and the result lies on the next diagonal. The figure below shows diagonal \\(n = 2\\): the values \\(C(2,0), C(3,1), C(4,2), C(5,3)\\) are connected in red and their sum points to \\(C(6,3) = 20\\) in amber.\nDiagonal Theorem: the sum along a diagonal gives the element on the next diagonal $$\\binom{2}{0} + \\binom{3}{1} + \\binom{4}{2} + \\binom{5}{3} = 1 + 3 + 6 + 10 = 20 = \\binom{6}{3} \\checkmark$$ Worked Examples # Symmetry and the complexity of exhaustive search # Problem: An algorithm evaluates all subsets of size \\(k = 3\\) and all those of size \\(k = 17\\) of a set with \\(n = 20\\) elements. How many subsets are evaluated in each case? What does symmetry imply?\nSolution:\n$$\\binom{20}{3} = \\frac{20 \\times 19 \\times 18}{3!} = \\frac{6{,}840}{6} = \\mathbf{1{,}140}$$$$\\binom{20}{17} = \\binom{20}{20-17} = \\binom{20}{3} = \\mathbf{1{,}140}$$The symmetry \\(\\binom{n}{r} = \\binom{n}{n-r}\\) has a direct practical consequence: evaluating subsets of size \\(r\\) costs exactly the same as evaluating their complements of size \\(n - r\\). The count is largest for \\(r\\) near \\(n/2\\) and smallest for \\(r\\) near \\(0\\) or \\(n\\). Therefore, algorithms that search for small or large subsets are more efficient than searches for subsets of medium size.\nSum of products of consecutive integers # Problem: Compute \\(S = 1 \\cdot 2 \\cdot 3 + 2 \\cdot 3 \\cdot 4 + \\cdots + 50 \\cdot 51 \\cdot 52\\).\nSolution: Note that \\(k(k+1)(k+2) = 6\\binom{k+2}{3}\\), since \\(\\binom{k+2}{3} = \\frac{(k+2)(k+1)k}{6}\\). Hence:\n$$S = 6\\sum_{k=1}^{50}\\binom{k+2}{3} = 6\\left[\\binom{3}{3} + \\binom{4}{3} + \\cdots + \\binom{52}{3}\\right]$$By the Column Theorem with \\(r = 3\\), summing from \\(k=3\\) to \\(52\\):\n$$\\binom{3}{3} + \\cdots + \\binom{52}{3} = \\binom{53}{4} = \\frac{53 \\times 52 \\times 51 \\times 50}{24} = 292{,}825$$$$S = 6 \\times 292{,}825 = \\mathbf{1{,}756{,}950}$$ Python: generating the triangle with dynamic programming Dynamic programming is a technique that solves a complex problem by splitting it into smaller subproblems and storing the intermediate results to reuse them, instead of recomputing them from scratch at every step. Pascal\u0026rsquo;s triangle is a classic example: \\(C(n,r)\\) depends on \\(C(n-1,r-1)\\) and \\(C(n-1,r)\\), which were already computed in the previous row. Pascal\u0026rsquo;s rule \\(\\binom{i}{j} = \\binom{i-1}{j-1} + \\binom{i-1}{j}\\) translates directly into code: each cell is the sum of the two above it. This avoids computing factorials and the risk of overflow for large \\(n\\).\ndef pascal(n): tri = [[1] * (i + 1) for i in range(n + 1)] for i in range(2, n + 1): for j in range(1, i): tri[i][j] = tri[i - 1][j - 1] + tri[i - 1][j] return tri for row in pascal(7): print(row) [1] [1, 1] [1, 2, 1] [1, 3, 3, 1] [1, 4, 6, 4, 1] [1, 5, 10, 10, 5, 1] [1, 6, 15, 20, 15, 6, 1] [1, 7, 21, 35, 35, 21, 7, 1] The initialization [1] * (i + 1) fills the borders with 1; the inner loop overwrites only the interior elements of each row. Complexity: \\(O(n^2)\\) in time and space, the same cost as filling in the table by hand.\nSQL: computing \\(C(n,r)\\) with WITH RECURSIVE A recursive common table expression (CTE) builds the table of factorials up to \\(15!\\); then, three joins compute \\(C(n,r) = \\frac{n!}{r! (n-r)!}\\) for all \\(0 \\leq r \\leq n \\leq 8\\).\nWITH RECURSIVE fact(n, f) AS ( SELECT 0, 1 UNION ALL SELECT n + 1, f * (n + 1) FROM fact WHERE n \u0026lt; 15 ) SELECT a.n AS n, b.n AS r, a.f / (b.f * c.f) AS \u0026#34;C(n,r)\u0026#34; FROM fact a -- a.n = row, a.f = n! JOIN fact b ON b.n \u0026lt;= a.n -- b.n = column r, b.f = r! JOIN fact c ON c.n = a.n - b.n -- c.f = (n-r)! WHERE a.n \u0026lt;= 8 ORDER BY a.n, b.n; The result is a table with one row per pair \\((n, r)\\), equivalent to the triangle read row by row. Compatible with SQLite, PostgreSQL, and DuckDB.\nSummary Table # The three sum theorems share the same structure: in each one, one parameter stays fixed while the other varies along a row, column, or diagonal of the triangle. The panel below highlights this pattern with the fixed parameter in red:\nThe three sum theorems differ only in the summation axis in the triangle Identity / Theorem Formula Symmetry \\(\\binom{n}{r} = \\binom{n}{n-r}\\) Boundary \\(\\binom{n}{0} = \\binom{n}{n} = 1\\) Secondary \\(\\binom{n}{1} = \\binom{n}{n-1} = n\\) Pascal\u0026rsquo;s Rule (Stifel) \\(\\binom{n}{r} + \\binom{n}{r+1} = \\binom{n+1}{r+1}\\) Row Theorem \\(\\displaystyle\\sum_{k=0}^{n} \\binom{n}{k} = 2^n\\) Column Theorem \\(\\displaystyle\\sum_{k=r}^{n} \\binom{k}{r} = \\binom{n+1}{r+1}\\) Diagonal Theorem \\(\\displaystyle\\sum_{k=0}^{r} \\binom{n+k}{k} = \\binom{n+r+1}{r}\\) Exercises # The following exercises apply the identities of the article: combinatorial proofs, reading the triangle, and use of the Row and Column Theorems. They are ordered from simplest to most complex.\nExercise 1: Generalized Pascal identity Prove, using a combinatorial argument, that: $$\\binom{n+2}{p+2} = \\binom{n}{p} + 2\\binom{n}{p+1} + \\binom{n}{p+2}$$ Proof Consider the set \\(\\{a_1,\\ldots,a_n, x, y\\}\\) with \\(n+2\\) elements. We count subsets of size \\(p+2\\) by cases:\nNeither \\(x\\) nor \\(y\\): \\(\\binom{n}{p+2}\\) Exactly \\(x\\) or exactly \\(y\\) (2 cases): \\(2\\binom{n}{p+1}\\) Both \\(x\\) and \\(y\\): \\(\\binom{n}{p}\\) The three cases do not overlap and cover all subsets of size \\(p+2\\). Therefore, \\(\\binom{n+2}{p+2} = \\binom{n}{p} + 2\\binom{n}{p+1} + \\binom{n}{p+2}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 2: Eighth row of the triangle Build row \\(n=8\\) of the triangle from row \\(n=7\\): \\(1\\ 7\\ 21\\ 35\\ 35\\ 21\\ 7\\ 1\\).\nSolution Adding adjacent pairs and keeping 1 at the ends: $$1\\quad 8\\quad 28\\quad 56\\quad 70\\quad 56\\quad 28\\quad 8\\quad 1$$ Exercise 3: Number of elements from subsets If a set \\(A\\) has 512 subsets, what is \\(|A|\\)?\nSolution By the Row Theorem, a set with \\(n\\) elements has exactly \\(2^n\\) subsets. We just need to solve:\n\\(2^n = 512 = 2^9 \\Rightarrow n = \\mathbf{9}\\)\nExercise 4: Cocktails with 2 or more ingredients How many cocktails (mixtures of ≥ 2 ingredients) can be made with 7 distinct ingredients?\nSolution Subsets of size ≥ 2: \\(2^7 - \\binom{7}{0} - \\binom{7}{1} = 128 - 1 - 7 = \\mathbf{120}\\)\nExercise 5: Proof by the Column Theorem Prove that \\(\\displaystyle\\sum_{k=1}^n k = \\dfrac{n(n+1)}{2}\\).\nProof Write \\(k = \\binom{k}{1}\\): $$\\sum_{k=1}^n \\binom{k}{1} = \\binom{n+1}{2} = \\frac{(n+1)n}{2}$$Therefore, \\(\\displaystyle\\sum_{k=1}^n k = \\dfrac{n(n+1)}{2}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nNext steps # The identities in this article reveal the internal structure of \\(\\binom{n}{r}\\), but they do not explain why this coefficient appears in products of powers. When we expand \\((a+b)^n\\), the coefficients that appear in each term \\(a^k b^{n-k}\\) are exactly \\(\\binom{n}{k}\\), and the Row Theorem is nothing more than the case \\(a = b = 1\\) of this expansion. This is the binomial theorem, the topic of the next article in the series.\n","date":"2026-07-23","externalUrl":null,"permalink":"/2026/07/binomial-coefficients/","section":"Blog","summary":"A set with 10 elements has 1,024 subsets, and that is the sum of row 10 of Pascal’s triangle. We will see the identities of binomial coefficients: symmetry, Pascal’s rule (Stifel’s rule), and the row, column, and diagonal theorems.","title":"Binomial Coefficients and Pascal's Triangle","type":"posts"},{"content":"","date":"2026-07-21","externalUrl":null,"permalink":"/tags/combinacoes/","section":"tags","summary":"","title":"combinações","type":"tags"},{"content":"","date":"2026-07-21","externalUrl":null,"permalink":"/tags/combinations/","section":"tags","summary":"","title":"combinations","type":"tags"},{"content":" At an ice cream shop with 7 flavors, how many different ways are there to choose 4 scoops, regardless of their order on the cone and allowing the same flavor more than once? This is a problem of combination with repetition.\nIn the article on arrangements with repetition, we saw that when order matters and repetition is allowed, the count is \\(n^r\\). Now we reach the last of the six types of grouping in the series: order no longer matters, but repetition is still allowed. The formula changes, and the technique for deriving it is elegant enough to have a name of its own: stars and bars.\nWhy combinations with repetition? # Selecting elements with repetition without regard to order is a central operation in computing and statistics:\nMultisets in programming: Python\u0026rsquo;s collections.Counter stores multisets, collections where an element can appear more than once, with no regard to order. The number of multisets of size \\(r\\) over a vocabulary of \\(n\\) elements is exactly \\(CR(n, r)\\). Distribution of identical resources: allocating \\(r\\) identical tokens, credits, or tasks among \\(n\\) processes or servers, regardless of the delivery sequence, is a \\(CR(n, r)\\) problem, the basis of scheduling and load balancing models. Histogram analysis: the number of possible histograms with \\(r\\) samples in \\(n\\) categories (only the counts per category, without recording which sample fell where) is \\(CR(n, r)\\). Integer programming: counting nonnegative integer solutions of \\(x_1 + x_2 + \\cdots + x_n = r\\) is equivalent to \\(CR(n, r)\\), a direct connection with optimization problems and state-space analysis. Information theory: grouping the sequences of \\(r\\) symbols from an alphabet of \\(n\\) by the frequency of each symbol, ignoring positions, we get exactly \\(CR(n, r)\\) groups, the types of sequence, used in proofs about data compression. From combination to combination with repetition # The diagram below completes the decision tree of the series, adding the sixth and last type of classic grouping:\n\u0026#x2197; flowchart TD A[\"n types of elements\"] --\u003e B{\"Repetition\nallowed?\"} B --\u003e|\"No: all distinct\"| C{\"Use all?\"} C --\u003e|\"Yes\"| D[\"Permutation\nP_n = n!\"] C --\u003e|\"No: choose r\"| E{\"Does order matter?\"} E --\u003e|\"Yes\"| F[\"Arrangement\nA(n,r) = n!/(n-r)!\"] E --\u003e|\"No\"| G[\"Combination\nC(n,r) = n!/(r!(n-r)!)\"] B --\u003e|\"Yes: with repetition\"| H{\"Use all?\"} H --\u003e|\"Yes: fixed multiset\"| I[\"Permutation with Repetition\nn!/(n₁!·n₂!·…·nₖ!)\"] H --\u003e|\"No: choose r\"| K{\"Does order matter?\"} K --\u003e|\"Yes\"| J[\"Arrangement with Repetition\nAR(n,r) = nʳ\"] K --\u003e|\"No\"| L[\"Combination with Repetition\nCR(n,r) = C(n+r-1,r)\"] Combination with Repetition # Definition A combination with repetition of \\(n\\) elements taken \\(r\\) at a time, hereafter denoted \\(CR(n,r)\\), is a selection of \\(r\\) elements without regard to order, where each element can appear more than once: $$CR(n, r) = \\binom{n + r - 1}{r} = \\binom{n + r - 1}{n - 1}$$Note: unlike the combination without repetition, \\(r\\) can be greater than \\(n\\), since we can select more elements than there are types available, simply by repeating some of them.\nThe Stars and Bars Method # How do we derive the formula? The idea is to encode each distribution as a sequence of two kinds of symbols, ★ (star) and | (bar), and count those sequences. The method appears in William Feller\u0026rsquo;s classic probability book, An Introduction to Probability Theory and Its Applications (1950), which helped make it known (see the entry Stars and bars). ★ and | form an alphabet of two symbols, exactly like 1 and 0 in a binary sequence: ★ stands for \u0026ldquo;one unit allocated\u0026rdquo; and | stands for \u0026ldquo;change of box\u0026rdquo;. In the article on permutations with repetition, we saw that arranging \\(k\\) symbols of one kind and \\(m\\) of another produces \\(\\binom{k+m}{k}\\) distinct sequences: that is the result we will use here.\nMotivating example: distributing 5 identical balls into 3 numbered boxes. Each distribution corresponds to a sequence of 5 stars (★) and 2 bars (|), where the bars separate the boxes:\n$$\\underbrace{★★}_{C_1} | \\underbrace{★}_{C_2} | \\underbrace{★★}_{C_3} \\quad\\longrightarrow\\quad ★ ★ | ★ | ★ ★$$The sequence has 5 stars and 2 bars, 7 symbols in total. Counting the distributions amounts to choosing which 2 positions (among the 7) receive a bar:\n$$C(7, 2) = C(5+3-1, 3-1) = \\binom{7}{2} = 21$$Generalization: \\(r\\) stars in \\(n\\) boxes use \\(n-1\\) bars, forming a sequence of \\(r + n - 1\\) symbols. Choosing the \\(r\\) positions of the stars (or the \\(n-1\\) of the bars) gives:\n$$CR(n, r) = \\binom{n + r - 1}{r}$$ Three Equivalent Formulations # The beauty of the stars and bars method is that it reveals that three problems with very different statements are, in fact, the same counting problem:\nClassic equivalences The three problems below have exactly \\(CR(n,r) = \\binom{n+r-1}{r}\\) solutions:\nSelection with repetition: choosing \\(r\\) elements from \\(n\\) types (regardless of order, repetition allowed). Distribution of identical objects: placing \\(r\\) indistinguishable balls into \\(n\\) distinct boxes (boxes may stay empty). Integer equations: nonnegative integer solutions of \\(x_1 + x_2 + \\cdots + x_n = r\\). The figure below takes the sequence from the motivating example, ★★|★|★★, and reads it in the three ways:\nA sequence of stars and bars, three problems Recognizing which of the three readings fits the statement is the central skill for solving problems with \\(CR\\).\nWorked Examples # 4 scoops from 7 flavors # Problem: An ice cream shop offers 7 flavors. In how many different ways can you order 4 scoops, regardless of order and allowing the same flavor to repeat?\nSolution: The order of the scoops on the cone does not matter (chocolate + strawberry = strawberry + chocolate), and repetition is allowed (four scoops of chocolate is a valid option). We apply the combination with repetition with \\(n = 7\\) flavors and \\(r = 4\\) scoops:\n$$CR(7, 4) = \\binom{7 + 4 - 1}{4} = \\binom{10}{4} = \\mathbf{210}$$Back to the opening example: there are \\(\\mathbf{210}\\) possible distinct orders.\nPython: itertools.combinations_with_replacement Python directly offers the function that generates combinations with repetition, itertools.combinations_with_replacement(iterable, r):\nfrom itertools import combinations_with_replacement from math import comb flavors = [\u0026#34;chocolate\u0026#34;, \u0026#34;strawberry\u0026#34;, \u0026#34;vanilla\u0026#34;, \u0026#34;lemon\u0026#34;, \u0026#34;passion fruit\u0026#34;, \u0026#34;grape\u0026#34;, \u0026#34;coconut\u0026#34;] # 7 flavors # Generates all orders of 4 scoops orders = list(combinations_with_replacement(flavors, 4)) print(len(orders)) # 210 = CR(7, 4) print(orders[0]) # (\u0026#39;chocolate\u0026#39;, \u0026#39;chocolate\u0026#39;, \u0026#39;chocolate\u0026#39;, \u0026#39;chocolate\u0026#39;) print(orders[-1]) # (\u0026#39;coconut\u0026#39;, \u0026#39;coconut\u0026#39;, \u0026#39;coconut\u0026#39;, \u0026#39;coconut\u0026#39;) # Check with the formula print(comb(7 + 4 - 1, 4)) # 210 Note that combinations_with_replacement returns each tuple with the flavors in the order they appear in the input list: that is why the first order is all chocolate and the last, all coconut. This canonical order is what guarantees that two orders with the same flavors in a different sequence show up as a single tuple.\n5 balls in 3 boxes # Problem: In how many ways can 5 identical balls be distributed into 3 distinct boxes (boxes may stay empty)?\nSolution: Identical balls in distinct boxes is the second reading of the classic equivalences, with \\(n = 3\\) boxes and \\(r = 5\\) balls.\n$$CR(3, 5) = \\binom{3 + 5 - 1}{5} = \\binom{7}{5} = \\binom{7}{2} = \\mathbf{21}$$The stars and bars method confirms it: 5 stars and 2 bars, choosing 2 positions among 7.\nWhy are integer equations solved with CR? The connection becomes clear when we name the boxes of the previous example with variables. Let \\(x_1\\) be the number of balls in box 1, \\(x_2\\) the number in box 2, and \\(x_3\\) the number in box 3. Each valid distribution is exactly one nonnegative integer solution of: $$x_1 + x_2 + x_3 = 5, \\quad x_i \\geq 0$$ For example, the distribution ★★ | ★ | ★★ corresponds to the solution \\((x_1, x_2, x_3) = (2, 1, 2)\\). The sequence of stars and bars ★ ★ | ★ | ★ ★ and the equation \\(2 + 1 + 2 = 5\\) carry the same information: they are two notations for the same object.\nThis generalizes immediately: nonnegative integer solutions of \\(x_1 + x_2 + \\cdots + x_n = r\\) are the same as distributions of \\(r\\) balls into \\(n\\) boxes, and are therefore counted by \\(CR(n, r)\\). The variable \\(x_i\\) is the \u0026ldquo;counter\u0026rdquo; of box \\(i\\); the equation is just the requirement that the total number of balls be \\(r\\).\nLikewise, the ice cream example can be written with variables: if \\(s_i\\) is the number of scoops of flavor \\(i\\) ordered, counting orders of 4 scoops from 7 flavors is the same as counting solutions of \\(s_1 + s_2 + \\cdots + s_7 = 4\\) with \\(s_i \\geq 0\\), and the result is \\(CR(7, 4) = 210\\), as we have already seen.\nInteger solutions of \\(x + y + z + w = 9\\) # Problem: How many nonnegative integer solutions does the equation \\(x + y + z + w = 9\\) have?\nSolution: We have \\(n = 4\\) variables (equivalent to 4 boxes) and \\(r = 9\\) (equivalent to 9 balls to distribute):\n$$CR(4, 9) = \\binom{4 + 9 - 1}{9} = \\binom{12}{9} = \\binom{12}{3} = \\mathbf{220}$$ Solutions of \\(x_1 + x_2 + x_3 = 10\\) with \\(x_2 \\geq 4\\) # Problem: How many nonnegative integer solutions does \\(x_1 + x_2 + x_3 = 10\\) have with the restriction \\(x_2 \\geq 4\\)?\nSolution: The restriction \\(x_2 \\geq 4\\) is handled by substitution: we define \\(y_2 = x_2 - 4 \\geq 0\\), which \u0026ldquo;absorbs\u0026rdquo; the required minimum of \\(x_2\\). The equation becomes:\n$$x_1 + (y_2 + 4) + x_3 = 10 \\implies x_1 + y_2 + x_3 = 6$$Now we have \\(n = 3\\) variables and \\(r = 6\\), all nonnegative:\n$$CR(3, 6) = \\binom{3 + 6 - 1}{6} = \\binom{8}{6} = \\binom{8}{2} = \\mathbf{28}$$ Solutions of \\(x + y + z + w \\leq 5\\) # Problem: How many nonnegative integer solutions does \\(x + y + z + w \\leq 5\\) have?\nSolution (Method 1, direct sum by cases): we split by total value \\(k = 0, 1, \\ldots, 5\\) and apply the sum rule:\n$$\\sum_{k=0}^{5} CR(4, k) = \\binom{3}{0} + \\binom{4}{1} + \\binom{5}{2} + \\binom{6}{3} + \\binom{7}{4} + \\binom{8}{5} = 1 + 4 + 10 + 20 + 35 + 56 = 126$$Solution (Method 2, slack variable): we introduce \\(f = 5 - (x+y+z+w) \\geq 0\\), which turns the inequality into an equality:\n$$x + y + z + w + f = 5 \\implies CR(5, 5) = \\binom{9}{5} = \\mathbf{126} \\checkmark$$The slack variable method is more elegant: instead of adding up 6 terms, it is solved with a single application of the formula.\nSummary Table # The table below gathers all six types of grouping in the series and the central result of this article:\nWithout repetition With repetition Uses all \\(n\\) elements Permutation: \\(n!\\) Permutation w/ Repetition: \\(\\dfrac{n!}{n_1!\\cdots n_k!}\\) Chooses \\(r\\), order matters Arrangement: \\(\\dfrac{n!}{(n-r)!}\\) Arrangement w/ Repetition: \\(n^r\\) Chooses \\(r\\), order does not matter Combination: \\(\\dfrac{n!}{r!(n-r)!}\\) Combination w/ Repetition: \\(\\dbinom{n+r-1}{r}\\) Specific results of this article:\nResult Formula Combination with repetition \\(CR(n,r) = \\dbinom{n+r-1}{r}\\) Equivalence with distribution of identical objects \\(r\\) balls in \\(n\\) boxes → \\(CR(n,r)\\) Equivalence with nonnegative integer solutions \\(x_1 + \\cdots + x_n = r, x_i \\geq 0\\) → \\(CR(n,r)\\) Inequality: \\(x_1 + \\cdots + x_n \\leq r, x_i \\geq 0\\) via slack variable \\(CR(n+1, r)\\) Exercises # The exercises below go through the three equivalent formulations of \\(CR(n,r)\\): selection with repetition, distribution of identical objects, and solutions of equations with restrictions. They are ordered from simplest to most complex.\nExercise 1: Distributing oranges In how many ways can we distribute 6 identical oranges among 2 people (each person may end up with no oranges)?\nSolution The oranges are identical (their order does not matter) and each person can receive any amount, including zero. This is equivalent to counting nonnegative integer solutions of \\(a + b = 6\\), with \\(n = 2\\) and \\(r = 6\\): $$CR(2, 6) = \\binom{7}{6} = \\binom{7}{1} = \\mathbf{7}$$ The distributions are: (0,6), (1,5), (2,4), (3,3), (4,2), (5,1), (6,0). Exactly 7. ✓\nExercise 2: Party sweets A candy shop offers 8 varieties of sweets. In how many ways can you buy 12 sweets, repeating varieties allowed and regardless of order?\nSolution Selection with repetition: \\(n = 8\\) varieties, \\(r = 12\\) sweets: $$CR(8, 12) = \\binom{8 + 12 - 1}{12} = \\binom{19}{12} = \\binom{19}{7} = \\mathbf{50{,}388}$$Here \\(r = 12\\) is greater than \\(n = 8\\), something the combination with repetition allows, as we saw in the note right after the definition: just repeat varieties.\nExercise 3: Balls in nonempty boxes In how many ways can 20 identical balls be placed into 5 distinct boxes with no box left empty?\nSolution The restriction \u0026ldquo;no empty box\u0026rdquo; means each box receives at least 1 ball. We make the substitution \\(x_i^\\ast = x_i - 1 \\geq 0\\), which absorbs the minimum of 1 per box. The total sum decreases by 5: $$x_1 + \\cdots + x_5 = 20, x_i \\geq 1 \\implies x_1^* + \\cdots + x_5^* = 15, x_i^* \\geq 0$$ $$CR(5, 15) = \\binom{5 + 15 - 1}{15} = \\binom{19}{15} = \\binom{19}{4} = \\mathbf{3{,}876}$$ Exercise 4: Sum of digits How many integers between 1 and 100,000 have digit sum equal to 6?\nSolution Every integer in this range can be represented as a sequence of 5 digits \\(d_1 d_2 d_3 d_4 d_5\\) with \\(d_i \\geq 0\\) (including leading zeros: 6 becomes 00006). The number 100,000 has digit sum 1, so it is not counted. We want nonnegative integer solutions of \\(d_1 + d_2 + d_3 + d_4 + d_5 = 6\\). Since the sum is 6 and no digit can exceed 9, the restriction \\(d_i \\leq 9\\) is satisfied automatically: $$CR(5, 6) = \\binom{5 + 6 - 1}{6} = \\binom{10}{6} = \\binom{10}{4} = \\mathbf{210}$$ Exercise 5: Positive integer solutions with an inequality How many positive integer solutions (\\(x, y, z \\geq 1\\)) does \\(x + y + z \u0026lt; 10\\) have?\nSolution Step 1 (eliminate the \\(\\geq 1\\)): let \\(x^\\ast = x - 1, y^\\ast = y - 1, z^\\ast = z - 1 \\geq 0\\). The inequality becomes \\(x^\\ast + y^\\ast + z^\\ast \u0026lt; 7\\), that is, \\(\\leq 6\\).\nStep 2 (turn the inequality into an equality): we introduce a slack variable \\(u = 6 - (x^\\ast + y^\\ast + z^\\ast) \\geq 0\\): $$x^* + y^* + z^* + u = 6, \\quad x^*, y^*, z^*, u \\geq 0$$Now we have \\(n = 4\\) variables and \\(r = 6\\): $$CR(4, 6) = \\binom{4 + 6 - 1}{6} = \\binom{9}{6} = \\binom{9}{3} = \\mathbf{84}$$ Next steps # The combination with repetition completes the triad of groupings with repetition and closes the picture of six types of counting in the series. The limitation of all these tools is that they work with simple counts: how many are there? The natural next step is to understand how these coefficients relate to one another: the identities that connect them, such as Pascal\u0026rsquo;s rule and the binomial theorem, topics of the next articles in the series.\n","date":"2026-07-21","externalUrl":null,"permalink":"/2026/07/combinations-with-repetition/","section":"Blog","summary":"At an ice cream shop with 7 flavors, there are 210 ways to order 4 scoops, repeating flavors allowed. We get there with the stars and bars method, which shows that choosing with repetition, distributing balls into boxes, and solving integer equations are the same problem.","title":"Combinations with Repetition: Stars and Bars","type":"posts"},{"content":"","date":"2026-07-16","externalUrl":null,"permalink":"/tags/arrangements/","section":"tags","summary":"","title":"arrangements","type":"tags"},{"content":" A 4-digit password allows repetition: the code 1111 is valid. How many 4-digit passwords exist? \\(10^4 = 10{,}000\\). This is the concept of an arrangement with repetition.\nIn the article on permutations with repetition, we saw how to count orderings of a fixed multiset, where the repeated elements are already given. Now the problem changes: we have \\(n\\) types of elements available and \\(r\\) positions to fill, and we may use any type in each position, as many times as we like.\nWhy arrangements with repetition? # Counting ordered sequences with repetition is one of the most recurrent operations in computing and mathematics:\nDigital storage systems: the number of values that \\(r\\) bits can represent is \\(AR(2, r) = 2^r\\), the basis for computing memory capacity, addressing, and state spaces in hardware. Cryptography and passwords: passwords of length \\(r\\) over an alphabet of \\(n\\) symbols (with repetition) total \\(n^r\\) possibilities, the formula that sizes resistance to brute-force attacks. Bioinformatics: DNA sequences of length \\(r\\) over the 4 bases (A, T, C, G) form \\(AR(4, r) = 4^r\\) sequences, the starting point for estimating sequence diversity in a genome. Network protocols: IPv4 addresses are 32-bit sequences, which gives \\(AR(2, 32) = 2^{32} \\approx 4.3\\) billion possible addresses. Number systems: the number of patterns representable with \\(r\\) digits in base \\(b\\) is \\(AR(b, r) = b^r\\), unifying binary, decimal, and hexadecimal in a single formula. From permutation with repetition to arrangement with repetition # The diagram below expands the decision tree from the previous articles, adding the branch that covers elements with repetition and partial selection:\n\u0026#x2197; flowchart TD A[\"n types of elements\"] --\u003e B{\"Repetition\nallowed?\"} B --\u003e|\"No: all distinct\"| C{\"Use all?\"} C --\u003e|\"Yes\"| D[\"Permutation\nP_n = n!\"] C --\u003e|\"No: choose r\"| E{\"Does order matter?\"} E --\u003e|\"Yes\"| F[\"Arrangement\nA(n,r) = n!/(n-r)!\"] E --\u003e|\"No\"| G[\"Combination\nC(n,r) = n!/(r!(n-r)!)\"] B --\u003e|\"Yes: with repetition\"| H{\"Use all?\"} H --\u003e|\"Yes: fixed multiset\"| I[\"Permutation with Repetition\nn!/(n₁!·n₂!·…·nₖ!)\"] H --\u003e|\"No: choose r positions\"| J[\"Arrangement with Repetition\nAR(n,r) = nʳ\"] Arrangement with Repetition # Definition An arrangement with repetition of \\(n\\) elements taken \\(r\\) at a time is an ordered sequence of \\(r\\) elements in which each element may be chosen more than once: $$AR(n, r) = n^r$$Justification: each of the \\(r\\) positions has \\(n\\) independent choices. By the product rule: \\(\\underbrace{n \\times n \\times \\cdots \\times n}_{r} = n^r\\).\nKey difference: in the arrangement without repetition, \\(r \\leq n\\). In the arrangement with repetition, \\(r\\) can be greater than \\(n\\), and we can fill more positions than there are available types, thanks to repetition.\nReading It as a Cartesian Product # If \\(A = \\{a_1, \\ldots, a_n\\}\\), the arrangements with repetition of \\(n\\) elements taken \\(r\\) at a time correspond to the elements of:\n$$\\underbrace{A \\times A \\times \\cdots \\times A}_{r \\text{ factors}}$$and therefore there are \\(|A|^r = n^r\\) arrangements. In Python, itertools.product(A, repeat=r) generates exactly this set.\nWorked Examples # Passwords of 2 symbols out of 8 available # Problem: A lock uses symbols from an alphabet of 8 characters. How many passwords of 2 symbols exist if repetition is allowed? And without repetition?\nSolution: With repetition, each position has 8 independent choices:\n$$AR(8, 2) = 8^2 = \\mathbf{64}$$This includes repeated pairs such as (A, A). Without repetition (simple arrangement), the 2nd position excludes the symbol already used: \\(A(8,2) = 8 \\times 7 = 56\\). The difference of 8 corresponds exactly to the 8 pairs of the form (Xᵢ, Xᵢ).\nPython: itertools.product for arrangements with repetition itertools.product(iterable, repeat=r) generates all arrangements with repetition of \\(r\\) elements of an iterable, exactly \\(n^r\\) tuples:\nfrom itertools import product, permutations symbols = \u0026#34;ABCDEFGH\u0026#34; # 8 symbols # AR(8, 2) = 8^2 = 64: all pairs with repetition allowed arrangements = list(product(symbols, repeat=2)) print(len(arrangements)) # 64 print(arrangements[:4]) # [(\u0026#39;A\u0026#39;, \u0026#39;A\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;D\u0026#39;)] # A(8, 2) = 56: without repetition without_rep = list(permutations(symbols, 2)) print(len(without_rep)) # 56 The difference between product(S, repeat=r) and permutations(S, r) is exactly the difference between arrangement with and without repetition: the first allows the same symbol to appear in more than one position; the second does not.\n3-digit sequences from \\(\\{1,2,3,4,5\\}\\) # Problem: How many 3-digit sequences can be formed with the elements of \\(\\{1,2,3,4,5\\}\\), allowing repetition?\nSolution: Since repetition is allowed, each position is filled independently of the others, and the choice for the 1st position does not restrict the options for the 2nd or the 3rd. By the product rule:\n$$AR(5, 3) = \\underbrace{5 \\times 5 \\times 5}_{3\\text{ positions}} = 5^3 = \\mathbf{125}$$Sequences such as (3, 3, 3) or (1, 5, 1) are valid and are all counted.\nVehicle license plates (3 letters + 4 digits, with repetition) # Problem: How many distinct vehicle license plates can be formed in the format of 3 letters followed by 4 digits, with repetition allowed?\nSolution: The choices of letters and digits are independent of each other. By the product rule:\n$$AR(26, 3) \\times AR(10, 4) = 26^3 \\times 10^4 = 17{,}576 \\times 10{,}000 = \\mathbf{175{,}760{,}000}$$ RGB color palette # Problem: Monitors and digital images represent colors as combinations of three channels: red (R), green (G), and blue (B). In the 8-bit-per-channel standard (called true color), each channel accepts intensities from 0 to 255, that is, 256 possible values. How many distinct colors can be represented? And on a display with 4 bits per channel, where each channel goes from 0 to 15?\nSolution: The three channels are chosen independently; there is repetition (two channels can have the same intensity, as in R = G = 128 for gray). By the product rule:\nFormat Intensities per channel Total colors 8 bits per channel (true color) 256 (from 0 to 255) \\(AR(256, 3) = 256^3 = 16{,}777{,}216\\) 4 bits per channel 16 (from 0 to 15) \\(AR(16, 3) = 16^3 = 4{,}096\\) The difference between the two formats is enormous: doubling the number of bits per channel multiplies the number of intensities by 16 (from 16 to 256) and the total number of colors by \\(16^3 = 4{,}096\\). The total goes from \\(4{,}096\\) to \\(4{,}096^2 = 16{,}777{,}216\\), that is, from a little over 4 thousand to almost 17 million colors.\nPython: enumerating colors and counting with itertools.product itertools.product(range(n), repeat=3) generates all (R, G, B) triples with intensities from 0 to n−1, exactly the \\(n^3\\) arrangements with repetition of 3 positions over an alphabet of \\(n\\) symbols:\nfrom itertools import product # Counting by formula for bits_per_channel in [4, 8]: n = 2 ** bits_per_channel # intensities per channel total = n ** 3 # AR(n, 3) total_fmt = f\u0026#34;{total:_}\u0026#34;.replace(\u0026#34;_\u0026#34;, \u0026#34;,\u0026#34;) # comma as thousands separator print(f\u0026#34;{bits_per_channel} bits/channel: {n} intensities → {total_fmt} colors\u0026#34;) # 4 bits/channel: 16 intensities → 4,096 colors # 8 bits/channel: 256 intensities → 16,777,216 colors # Generating the first colors of a 4-bit palette (n=16) palette_4bits = list(product(range(16), repeat=3)) print(len(palette_4bits)) # 4096 print(palette_4bits[:4]) # [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3)] # In CSS, colors are written as #RRGGBB in hexadecimal def to_hex(r, g, b): return f\u0026#34;#{r:02X}{g:02X}{b:02X}\u0026#34; print(to_hex(255, 128, 0)) # #FF8000 (orange) print(to_hex(0, 0, 0)) # #000000 (black) For the 8-bit-per-channel format, materializing the full list would take considerable memory (16 million triples). The formula \\(256^3\\) gives the total in constant time: this is when combinatorics replaces enumeration.\nComparison: Arrangement without vs. with Repetition # The table below summarizes the essential differences between the two kinds of arrangement:\nType Repetition? \\(r \u0026gt; n\\) allowed? Formula Arrangement without repetition No No \\(\\dfrac{n!}{(n-r)!}\\) Arrangement with repetition Yes Yes \\(n^r\\) Application: Computational Representation of Numbers # Every floating-point system stores a number as a sequence of positions, each filled with a symbol from a fixed alphabet. How many distinct numbers the system can represent is, therefore, a count of arrangements with repetition: the base of the alphabet raised to the number of positions.\nWhat varies between systems is the base used: modern computers use base 2 (binary), following the IEEE 754 standard; many scientific calculators of the 1970s and 1980s used base 10 (decimal), storing each digit directly in BCD (Binary Coded Decimal). The advantage of BCD is that the number shown on the display is identical to the number stored, avoiding the decimal ↔ binary conversion errors that affect IEEE 754 (such as the famous \\(0.1 + 0.2 \\neq 0.3\\)). The cost is space: BCD takes more memory than plain binary for the same range of values.\nThe two examples below illustrate this difference.\nNote For a complete treatment of the IEEE 754 standard and of rounding errors in binary, see the article Inside IEEE 754 and the article on base conversion for real numbers.\nIntel 8087 (binary floating point, 32-bit format):\nThe Intel 8087 coprocessor (1980) works internally with 80-bit numbers, but it also reads and writes the 32-bit and 64-bit formats that would later be standardized by IEEE 754. In the 32-bit single-precision format, each position is a bit, with an alphabet of 2 symbols (0 or 1). The total number of distinct patterns is:\n$$AR(2, 32) = 2^{32} = 4{,}294{,}967{,}296 \\text{ bit patterns}$$(Not all of them correspond to \u0026ldquo;useful\u0026rdquo; numbers: some bits encode the sign, the exponent, and the significand, and certain patterns represent special values such as infinity and NaN (Not a Number). The arrangement counts the patterns; interpreting them is the job of the IEEE 754 standard.)\nHP-11C calculator (decimal floating point):\nThe HP-11C (1981) uses BCD: each position stores a decimal digit (from 0 to 9), not a bit. Numbers have a 10-digit significand and a 2-digit exponent, each with its own sign, and the storage format has three independent parts:\nComponent Positions Alphabet Count Sign of the significand and of the exponent 2 (one each) \\(\\{+, -\\}\\) → 2 symbols \\(AR(2,2) = 2^2 = 4\\) Exponent (characteristic) 2 decimal digits \\(\\{0,\\ldots,9\\}\\) → 10 symbols \\(AR(10,2) = 10^2 = 100\\) Normalized significand 10 digits 1st digit from 1 to 9; others from 0 to 9 \\(9 \\times 10^9\\) Note The significand is normalized: the 1st digit is always nonzero (from 1 to 9), since a leading zero would be redundant, as the number could be rewritten with a smaller exponent. This reduces the options for the 1st digit from 10 to 9, exactly as in counting 3-digit numbers that exclude those with a leading zero.\nThe total number of distinct configurations is:\n$$4 \\times 10^2 \\times 9 \\times 10^9 = 36 \\times 10^{11} = 3{,}600{,}000{,}000{,}000$$That is 3.6 trillion configurations. Each digit and each sign takes 4 bits, and each HP-11C register has 14 such groups: the sign and the 10 digits of the significand, plus 3 groups for the exponent and its sign, which gives 56 bits in total. The same 56 bits, used as plain binary, would give \\(2^{56} \\approx 7.2 \\times 10^{16}\\) patterns, about 20 thousand times more. That is the cost of BCD: each decimal digit spends 4 bits, which could distinguish 16 values, to store only 10.\nThe figure below puts the two formats one over the other, on the same bit scale:\n32 binary bits and 56 BCD bits, on the same scale Summary Table # Concept Result Arrangements with repetition of \\(n\\) types taken \\(r\\) at a time \\(AR(n, r) = n^r\\) Arrangement without repetition \\(A(n,r) = \\dfrac{n!}{(n-r)!}\\) Can \\(r\\) exceed \\(n\\)? Yes, in the arrangement with repetition; no, in the one without Cartesian product \\(A \\times \\cdots \\times A\\) (\\(r\\) factors) \\(|A|^r = n^r\\) arrangements Exercises # The exercises below apply the formula \\(AR(n,r) = n^r\\) in different contexts: direct counting, counting with position restrictions, and the complement method. They are ordered from simplest to most complex.\nExercise 1: 3-digit numbers from \\(\\{2,3,5,8,9\\}\\) with repetition (a) Total. (b) Less than 800. (c) Multiples of 5. (d) Even. (e) Odd.\nSolution (a) With no restriction, each position can be any of the 5 elements: $$AR(5,3) = 5^3 = \\mathbf{125}$$(b) For the number to be less than 800, the hundreds digit must be less than 8. In the set \\(\\{2,3,5,8,9\\}\\), the elements less than 8 are \\(\\{2,3,5\\}\\), that is, 3 options. The other two positions remain free (5 options each): $$3 \\times 5 \\times 5 = 3 \\times 25 = \\mathbf{75}$$(c) A multiple of 5 ends in 0 or 5. From the set \\(\\{2,3,5,8,9\\}\\), only 5 satisfies this condition: the units digit is fixed. The other two positions have 5 options each: $$5 \\times 5 \\times 1 = \\mathbf{25}$$(d) An even number ends in an even digit. The even ones in the set are \\(\\{2,8\\}\\), that is, 2 options for the units digit. The other two positions have 5 options each: $$5 \\times 5 \\times 2 = \\mathbf{50}$$(e) By complement with respect to the total: even + odd = 125. Alternatively, the odd ones in the set are \\(\\{3,5,9\\}\\), that is, 3 options for the units digit: $$5 \\times 5 \\times 3 = \\mathbf{75}$$ Check: \\(50 + 75 = 125\\) ✓\nExercise 2: 5-letter words with A but not as the first letter With an alphabet of 26 letters and repetition allowed, how many 5-letter words contain A, but not as the first letter?\nSolution Directly counting \u0026ldquo;has A in at least one of positions 2 to 5\u0026rdquo; would require handling many overlapping cases. The nested complement strategy is cleaner:\nDefine two sets:\n\\(U\\) = words without A in the 1st position: the 1st position has 25 options (any letter except A); the other 4 positions accept any of the 26 letters. So \\(|U| = 25 \\times 26^4\\). \\(B\\) = words without A in any position: all 5 positions have 25 options. So \\(|B| = 25^5\\). Note that \\(B \\subset U\\): every word with no A anywhere also has no A in the 1st position. The difference \\(|U| - |B|\\) gives exactly the words without A in the 1st position but with A in some of positions 2 to 5, which is what the problem asks:\n$$|U| - |B| = 25 \\times 26^4 - 25^5 = 25(26^4 - 25^4) = \\mathbf{1{,}658{,}775}$$ Exercise 3: 5-digit numbers that contain the digit 2 (a) That contain the digit 2. (b) That do not contain the digit 2.\nSolution Total 5-digit numbers: the 1st digit cannot be zero (otherwise the number would have fewer than 5 digits), so it has 9 options (from 1 to 9); each of the other 4 digits has 10 options (from 0 to 9): $$9 \\times 10^4 = \\mathbf{90{,}000}$$(b) Without the digit 2: we remove 2 from the available alphabet. The 1st digit cannot be 0 or 2, leaving \\(\\{1,3,4,5,6,7,8,9\\}\\), that is, 8 options. Each of the other 4 digits can be any digit except 2, that is, \\(\\{0,1,3,4,5,6,7,8,9\\}\\), with 9 options each: $$8 \\times 9^4 = 8 \\times 6{,}561 = \\mathbf{52{,}488}$$(a) Directly counting the numbers with at least one digit 2 would require adding up many overlapping cases. By complement: $$90{,}000 - 52{,}488 = \\mathbf{37{,}512}$$ Exercise 4: Anagrams with and without repetition With \\(\\{a,b,c,d,e\\}\\), how many 3-letter \u0026ldquo;anagrams\u0026rdquo; exist: (a) With distinct letters. (b) With at least 2 equal letters.\nSolution (a) With distinct letters, no position can repeat the letter already used, which characterizes an arrangement without repetition: $$A(5,3) = 5 \\times 4 \\times 3 = \\mathbf{60}$$(b) Counting directly would require handling \u0026ldquo;exactly 2 equal\u0026rdquo; and \u0026ldquo;all 3 equal\u0026rdquo; separately. The complement is more direct: \\(AR(5,3)\\) counts all 3-letter sequences with repetition allowed; \\(A(5,3)\\) counts exactly those whose letters are all distinct. The difference is the sequences with at least 2 equal letters: $$AR(5,3) - A(5,3) = 125 - 60 = \\mathbf{65}$$ Next steps # Arrangements with repetition count ordered sequences where elements repeat. The limitation that remains is order: what about when it does not matter but repetition is allowed? This case, in which we select a subset of \\(r\\) elements with replacement among \\(n\\) available types, without regard to the sequence, leads to combinations with repetition, the topic of the next article in the series.\n","date":"2026-07-16","externalUrl":null,"permalink":"/2026/07/arrangements-with-repetition/","section":"Blog","summary":"A 4-digit password allows repetition, and that is why there are 10,000 of them: ten options in each of the four positions. The arrangement with repetition counts passwords, license plates, and RGB colors, and shows how many patterns fit in the 32 bits of a float and in the registers of an HP-11C calculator.","title":"Arrangements with Repetition: n^r and Computational Representation","type":"posts"},{"content":"","date":"2026-07-16","externalUrl":null,"permalink":"/tags/arranjos/","section":"tags","summary":"","title":"arranjos","type":"tags"},{"content":"","date":"2026-07-14","externalUrl":null,"permalink":"/tags/permutacoes/","section":"tags","summary":"","title":"permutações","type":"tags"},{"content":"","date":"2026-07-14","externalUrl":null,"permalink":"/tags/permutations/","section":"tags","summary":"","title":"permutations","type":"tags"},{"content":" How many anagrams does the word MAMBEMBE have? A plain \\(8!\\) overestimates: swapping the two letters B produces the same anagram, not a new one. When there are repeated elements, we need to divide to compensate.\nIn the article on combinations, we closed the classic triad of groupings with distinct elements. Now we take the next step: what happens when the elements repeat?\nWhy permutations with repetition? # Counting orderings with repeated elements shows up often in computing and data science:\nAnagrams: counting the distinct anagrams of a word with repeated letters (such as MISSISSIPPI) is the textbook problem for this formula. Bioinformatics: DNA sequences are made of four bases (A, T, C, G) that repeat. The number of distinct sequences of length \\(n\\) with a fixed composition is a permutation with repetition. Grid paths: the number of shortest paths between two points of an obstacle-free grid is a permutation with repetition of horizontal and vertical moves. Statistics: the multinomial coefficient \\(\\dfrac{n!}{n_1! n_2! \\cdots n_r!}\\) (which is exactly this formula) appears in the multinomial distribution, which generalizes the binomial distribution to more than two possible outcomes. From combination to permutation with repetition # The diagram below expands the decision tree from the previous article, adding the branch that covers repeated elements:\n\u0026#x2197; flowchart TD A[\"n elements\"] --\u003e B{\"All distinct?\"} B --\u003e|\"Yes\"| C{\"Use all?\"} C --\u003e|\"Yes\"| D[\"Permutation\nP_n = n!\"] C --\u003e|\"No: choose r\"| E{\"Does order matter?\"} E --\u003e|\"Yes\"| F[\"Arrangement\nA(n,r) = n!/(n-r)!\"] E --\u003e|\"No\"| G[\"Combination\nC(n,r) = n!/(r!(n-r)!)\"] B --\u003e|\"No: there are repeats\"| H[\"Permutation with Repetition\nn! / (n₁! · n₂! · … · nᵣ!)\"] The Central Idea: Removing the Overcount # Imagine 5 pens: 2 white (W¹ and W²), 1 blue, 1 black, 1 green. If we treat the two white ones as distinct, we get \\(5!\\) distributions. But W¹=white and W²=white are the same thing to whoever receives them: each real distribution was counted \\(2!\\) times.\nCorrect number: \\(\\dfrac{5!}{2!} = 60\\).\nPermutation with Repetition # Definition Given a set of \\(n\\) elements, of which \\(n_1\\) are equal to type 1, \\(n_2\\) to type 2, \u0026hellip;, \\(n_r\\) to type \\(r\\), with \\(n_1 + n_2 + \\cdots + n_r = n\\), the number of distinct arrangements (permutations) is: $$P_n^{(n_1, n_2, \\ldots, n_r)} = \\frac{n!}{n_1! n_2! \\cdots n_r!}$$Intuition: we start with the \\(n!\\) orderings of all the elements as if they were distinct. We divide by \\(n_i!\\) for each group of equal elements, which permute among themselves without creating anything new.\nWorked Examples # Numbers formed with 1,1,1,3,3,9 # Problem: How many distinct 6-digit numbers can be formed with the digits 1, 1, 1, 3, 3, 9?\nSolution:\nVia the formula:\n$$\\frac{6!}{3! 2! 1!} = \\frac{720}{12} = 60$$Via choosing positions: choose 3 positions (out of 6) for the 1s: \\(C(6,3) = 20\\). Then 2 positions (out of the 3 remaining) for the 3s: \\(C(3,2) = 3\\). The 9 takes the last one: \\(C(1,1) = 1\\). Total: \\(20 \\times 3 \\times 1 = 60\\) ✓\nAnagrams of MAMBEMBE # Letters: M (3×), B (2×), E (2×), A (1×), 8 letters in total.\n$$P_8^{(3,2,2,1)} = \\frac{8!}{3! 2! 2! 1!} = \\frac{40{,}320}{24} = \\mathbf{1{,}680}$$ Python: math.factorial and collections.Counter Python lets you compute the formula directly with math.factorial, or use sympy.factorial to work symbolically. The most idiomatic pattern combines collections.Counter to count the frequencies with math.prod to compute the denominator:\nfrom math import factorial from collections import Counter from math import prod def permutations_with_repetition(word: str) -\u0026gt; int: n = len(word) frequencies = Counter(word).values() return factorial(n) // prod(factorial(k) for k in frequencies) print(permutations_with_repetition(\u0026#34;MAMBEMBE\u0026#34;)) # 1680 print(permutations_with_repetition(\u0026#34;MISSISSIPPI\u0026#34;)) # 34650 Counter returns a dictionary {letter: count}, exactly the \\(n_i\\) of the formula. prod(factorial(k) for k in ...) computes the denominator without having to name each frequency individually.\nTournament with 15 matches: 7 wins, 5 losses, 3 draws # How many possible sequences of results are there?\n$$\\frac{15!}{7! 5! 3!} = \\mathbf{360{,}360}$$ Binary sequences with three 1s and five 0s # How many 8-bit sequences have exactly three 1s and five 0s?\n$$P_8^{(3,5)} = \\frac{8!}{3! 5!} = \\binom{8}{3} = \\mathbf{56}$$This result reveals an important equivalence:\nBinary sequences and the binomial coefficient Any sequence with \\(k\\) symbols of one type and \\(m\\) symbols of another type, regardless of what those symbols are, has exactly $$P_{k+m}^{(k, m)} = \\frac{(k+m)!}{k! m!} = \\binom{k+m}{k}$$ distinct arrangements. In the example above, \\(k = 3\\) (ones) and \\(m = 5\\) (zeros), giving \\(\\binom{8}{3} = 56\\). This is the binomial coefficient, which gets its own article later in this series. This formula reappears whenever a problem boils down to placing two types of symbols in sequence, like the stars and bars method, which will be covered in the article on combinations with repetition.\nPython: itertools.combinations for binary sequences Generating all binary sequences of length 8 with exactly three 1s is equivalent to choosing which 3 positions (out of 8) receive the value 1, that is, a combination. itertools.combinations does exactly that:\nfrom itertools import combinations from math import comb n, k = 8, 3 # Count directly through the equivalence with C(n, k) print(comb(n, k)) # 56 # Generate all sequences: each tuple gives the positions of the 1s positions = list(combinations(range(n), k)) print(len(positions)) # 56 # Convert to binary strings def to_binary(pos, n): seq = [\u0026#39;0\u0026#39;] * n for i in pos: seq[i] = \u0026#39;1\u0026#39; return \u0026#39;\u0026#39;.join(seq) sequences = [to_binary(p, n) for p in positions] print(sequences[:4]) # [\u0026#39;11100000\u0026#39;, \u0026#39;11010000\u0026#39;, \u0026#39;11001000\u0026#39;, \u0026#39;11000100\u0026#39;] Shortest grid paths: from (0,0) to (4,3) # Move only up (U) or to the right (R), in 7 steps (4R and 3U). The figure below shows two of the 35 possible paths: the blue path goes up first and then to the right; the yellow one does the opposite.\nTwo distinct paths among the 35 possible $$P_7^{(4,3)} = \\frac{7!}{4! 3!} = \\binom{7}{4} = \\mathbf{35} \\text{ paths}$$Why is the count the same as for anagrams? Because each shortest path is a 7-letter word with four R and three U, read from left to right as a list of steps, and each such word describes a path. The figure below shows this correspondence letter by letter for RURRURU, one of the 35 anagrams of RRRRUUU (the same four R and three U letters, in another order):\nRURRURU, one of the 35 anagrams of RRRRUUU, is one of the 35 paths from A to B Anagrams of MAMBEMBE starting with a vowel # Problem: How many anagrams of MAMBEMBE have a vowel in the first position?\nSolution: The available vowels are A, E, E. The cases are disjoint according to the initial vowel.\nCase 1 (starts with A): the remaining ones are M,M,M,B,B,E,E (7 letters):\n$$N_1 = \\frac{7!}{3! 2! 2!} = \\mathbf{210}$$Case 2 (starts with E): the remaining ones are M,M,M,A,B,B,E (7 letters):\n$$N_2 = \\frac{7!}{3! 1! 2! 1!} = \\mathbf{420}$$Total (sum rule, disjoint cases): \\(210 + 420 = \\mathbf{630}\\)\nAnagrams of MAMBEMBE with all the Ms apart # Problem: How many anagrams of MAMBEMBE have the three Ms in positions that are not adjacent to each other?\nSolution:\nStep 1: order the 5 non-M letters: A, B, B, E, E.\n$$\\frac{5!}{2! 2! 1!} = 30 \\text{ orderings}$$Step 2: each ordering has 6 gaps (before, between, and after the letters). Insert 3 Ms into 3 distinct gaps (so they are not adjacent):\n$$C(6, 3) = 20$$Total: \\(30 \\times 20 = \\mathbf{600}\\)\nSummary Table # Type Equal elements? Formula Permutation No \\(P_n = n!\\) Permutation with repetition Yes \\(\\dfrac{n!}{n_1! n_2! \\cdots n_r!}\\) Special case: two types Yes (two groups) \\(\\dfrac{n!}{k! (n-k)!} = \\dbinom{n}{k}\\) Exercises # The following exercises cover the main techniques of the article: direct application of the formula, cases with position restrictions, and grid paths. They are ordered from simplest to most complex.\nExercise 1: Numbers with repeated digits How many 7-digit numbers can be formed with \\(1, 3, 6, 6, 6, 8, 8\\)? (a) With no restrictions. (b) Greater than 6,000,000.\nSolution (a) \\(\\dfrac{7!}{3! 2! 1! 1!} = \\dfrac{5{,}040}{12} = \\mathbf{420}\\)\n(b) 1st digit ∈ {6, 8}.\nStarts with 6: the remaining ones are \\(6,6,8,8,1,3\\) → \\(\\dfrac{6!}{2! 2! 1! 1!} = 180\\) Starts with 8: the remaining ones are \\(6,6,6,8,1,3\\) → \\(\\dfrac{6!}{3! 1! 1! 1!} = 120\\) Total: \\(180 + 120 = \\mathbf{300}\\)\nExercise 2: 5-digit subsets of \\(\\{1,1,1,1,2,3\\}\\) How many 5-digit numbers can be formed by choosing 5 of the 6 digits \\(1,1,1,1,2,3\\)?\nSolution Disjoint groups (which 5 of the 6 we use):\n\\(\\{1,1,1,1,2\\}\\): \\(\\dfrac{5!}{4!} = 5\\) \\(\\{1,1,1,1,3\\}\\): \\(\\dfrac{5!}{4!} = 5\\) \\(\\{1,1,1,2,3\\}\\): \\(\\dfrac{5!}{3!} = 20\\) Total: \\(5 + 5 + 20 = \\mathbf{30}\\)\nThe next two exercises use the same map. The figure below shows the grid with points A, B, and C marked and, anticipating Exercise 4, highlights one possible path from A to C (in blue) and one from C to B (in yellow):\nGrid for exercises 3 and 4: point C splits the route A→B into two independent segments Exercise 3: Shortest grid paths On a map with 7 N-S (North-South, that is, vertical) avenues and 6 E-W (East-West, that is, horizontal) avenues, how many shortest routes are there from the bottom-left corner (A) to the top-right corner (B)?\nSolution With 7 N-S avenues there are 6 horizontal blocks to cross (6 moves to the right); with 6 E-W avenues there are 5 vertical blocks to cross (5 moves up). Any path that only moves forward (never backtracks) uses exactly these 11 steps, so 11 is the minimum length: $$\\frac{11!}{6! 5!} = C(11,6) = \\mathbf{462}$$ Exercise 4: Shortest paths through C On the same map, how many routes A→B pass through the point C located 4 blocks to the right and 4 up from A?\nSolution A→C: 4R + 4U = \\(\\dfrac{8!}{4! 4!} = 70\\) C→B: 2R + 1U = \\(\\dfrac{3!}{2! 1!} = 3\\) Total: \\(70 \\times 3 = \\mathbf{210}\\)\nNext steps # Permutation with repetition solves the case where we use all \\(n\\) elements, repeated ones included. The limitation that remains is this requirement of using all of them: what if we want to choose only \\(r \u0026lt; n\\) elements from a set with repetitions allowed, and order still matters?\nThis case, in which we select \\(r\\) elements with replacement among \\(n\\) available types, leads to arrangements with repetition, the topic of the next article in the series.\n","date":"2026-07-14","externalUrl":null,"permalink":"/2026/07/permutations-with-repetition/","section":"Blog","summary":"How many anagrams does the word MAMBEMBE have? Fewer than an 8-letter word with distinct letters would have, because swapping two equal letters does not create a new anagram. The formula discounts the repetitions by dividing by factorials, and it works for binary sequences and for shortest paths on grids.","title":"Permutations with Repetition: When Elements Repeat","type":"posts"},{"content":" A group of 10 friends goes to the movies and needs to pick 3 people to buy the popcorn. Only who goes matters, not the order in which they were chosen. This is a combination.\nWhy combinations? # Selecting subsets without regard to order is one of the most frequent operations in computing and data science:\nData analysis: the number of pairs of variables in a set of \\(n\\) columns is \\(C(n, 2)\\), the basis of correlation matrices and association tests in SQL (Structured Query Language) and pandas. Software testing: the all-pairs testing technique (testing every pair of parameters) must cover the interactions of each of the \\(C(k, 2)\\) pairs that can be formed from \\(k\\) parameters. Graph theory: the maximum number of edges in a simple graph with \\(n\\) vertices is \\(C(n, 2)\\), because each edge is a combination of two distinct vertices, with no direction. Cryptography: threshold schemes allow any subgroup of at least \\(r\\) out of \\(n\\) participants to reconstruct a secret key. The number of minimal subgroups that reconstruct it is \\(C(n, r)\\). Statistics and machine learning: the binomial coefficient \\(C(n, r)\\) appears in the binomial distribution and in feature subset selection. With \\(n\\) features, there are \\(C(n, r)\\) possible models that use exactly \\(r\\) of them. From arrangement to combination # In the article on arrangements, we saw that choosing \\(r\\) distinct elements out of \\(n\\) splits into two cases depending on whether order matters. The diagram below completes that decision tree by adding the combination branch:\n\u0026#x2197; flowchart TD A[\"n distinct elements\"] --\u003e B{\"Use all of them?\"} B --\u003e|\"Yes\"| C[\"Permutation\nP_n = n!\"] B --\u003e|\"No, choose r ≤ n\"| D{\"Does order matter?\"} D --\u003e|\"Yes\"| E[\"Arrangement\nA(n,r) = n!/(n-r)!\"] D --\u003e|\"No\"| F[\"Combination\nC(n,r) = n!/(r!(n-r)!)\"] Combinations # The diagram above already anticipates the formal definition:\nDefinition A combination of \\(n\\) distinct elements taken \\(r\\) at a time (an r-combination) is a selection of \\(r\\) elements without regard to order: $$C(n, r) = \\binom{n}{r} = \\frac{n!}{r! (n-r)!}$$The formula comes straight from the arrangement: each combination generates \\(r!\\) arrangements, one for each reordering of the \\(r\\) chosen elements. Dividing the arrangement count by the number of internal reorderings gives the combination:\n$$A(n,r) = C(n,r) \\cdot r! \\implies C(n,r) = \\frac{A(n,r)}{r!} = \\frac{n!}{r! (n-r)!}$$ Properties # Three fundamental identities simplify many practical calculations:\nSymmetry $$C(n, r) = C(n, n-r)$$ Choosing who joins the group is equivalent to choosing who stays out: the two subsets determine each other.\nBasic identities $$C(n, 0) = C(n, n) = 1$$ $$C(n, 1) = C(n, n-1) = n$$ Note When \\(r \u0026gt; n/2\\), use the symmetry \\(C(n, r) = C(n, n-r)\\): computing with \\(n - r \u0026lt; r\\) reduces the number of factors in the numerator and simplifies the calculation by hand.\nArrangement vs. Combination: How to Decide # The key question is always: \u0026ldquo;Does order change the result?\u0026rdquo; The table below summarizes the most common cases:\nSituation Does order matter? Technique Race podium (1st, 2nd, 3rd) Yes Arrangement Committee of 3 members out of 10 No Combination 4-digit password with distinct digits Yes Arrangement 5-card hand from a deck No Combination Worked Examples # Committee of 3 out of 10 people # Problem: An association needs to choose 3 members out of 10 volunteers to form a committee. How many distinct committees can be formed?\nSolution: Order does not matter, since a committee \\(\\{A, B, C\\}\\) is the same as \\(\\{B, A, C\\}\\). We apply the combination formula:\n$$C(10, 3) = \\frac{10!}{3! 7!} = \\frac{10 \\times 9 \\times 8}{6} = \\mathbf{120}$$Back to the opening example: the group of 10 friends can choose the 3 people for the popcorn in 120 distinct ways.\nSQL: JOIN with an order filter to generate combinations In SQL, the difference between an arrangement and a combination is just the join condition. In the article on arrangements we used \u0026lt;\u0026gt; to avoid repetition while generating both directions; here we use \u0026gt; to impose a canonical order and count each subset exactly once:\nCREATE TABLE people (id INT, name TEXT); INSERT INTO people VALUES (1,\u0026#39;Ana\u0026#39;),(2,\u0026#39;Bruno\u0026#39;),(3,\u0026#39;Carlos\u0026#39;),(4,\u0026#39;Diana\u0026#39;),(5,\u0026#39;Eva\u0026#39;), (6,\u0026#39;Felipe\u0026#39;),(7,\u0026#39;Gabi\u0026#39;),(8,\u0026#39;Hugo\u0026#39;),(9,\u0026#39;Iara\u0026#39;),(10,\u0026#39;João\u0026#39;); -- id_a \u0026lt; id_b \u0026lt; id_c guarantees that each trio appears exactly once, -- regardless of the order of the names SELECT a.name, b.name, c.name FROM people a JOIN people b ON b.id \u0026gt; a.id JOIN people c ON c.id \u0026gt; b.id ORDER BY a.name, b.name, c.name; SELECT COUNT(*) FROM people a JOIN people b ON b.id \u0026gt; a.id JOIN people c ON c.id \u0026gt; b.id; -- 120 = C(10, 3) Replacing \u0026gt; with \u0026lt;\u0026gt; in the join conditions and adding AND c.id \u0026lt;\u0026gt; a.id (so that the 1st and the 3rd are also distinct), we would count the arrangements \\(A(10, 3) = 720\\), six times as many, since each trio would appear in all \\(3! = 6\\) possible orders.\n5-card hand from a 52-card deck # Problem: How many hands of 5 distinct cards can be dealt from a standard 52-card deck?\nSolution: The order of the cards in the hand does not matter, since a hand \\(\\{A\\spadesuit, K\\heartsuit, Q\\diamondsuit, J\\clubsuit, 10\\spadesuit\\}\\) is the same in any sequence in which the cards arrive:\n$$C(52, 5) = \\frac{52!}{5! 47!} = 2{,}598{,}960$$ Python: itertools.combinations and math.comb Python offers two ways to work with combinations: math.comb(n, r) to compute the formula, and itertools.combinations(iterable, r) to generate each subset individually.\nfrom itertools import combinations import math # math.comb computes C(n, r) directly: fast, without generating all cases print(math.comb(52, 5)) # 2598960 # itertools.combinations generates each hand: useful to filter by property # (e.g., how many hands have at least one ace?) deck = range(52) # 52 cards represented by 0-51 hands = combinations(deck, 5) print(sum(1 for _ in hands)) # 2598960 # For large n and r, NEVER materialize the iterator into a list: # use math.comb for the total and combinations() only when you need to inspect itertools.combinations guarantees that each element appears at most once per subset and that subsets with the same elements in different orders are treated as equal, exactly the definition of a combination.\nSoccer team: 11 players out of 15 # Problem: A coach needs to pick 11 starters out of 15 available players. How many distinct lineups (ignoring positions) are there?\nSolution: Since \\(r = 11 \u0026gt; 15/2\\), the symmetry property simplifies the calculation, because choosing the 11 who play is equivalent to choosing the 4 who sit out:\n$$C(15, 11) = C(15, 4) = \\frac{15 \\times 14 \\times 13 \\times 12}{4!} = \\frac{32{,}760}{24} = \\mathbf{1{,}365}$$ Mixed group: 3 men and 2 women out of 6 men and 5 women # Problem: A committee of 5 people must have exactly 3 men and 2 women. There are 6 men and 5 women available. How many distinct committees can be formed?\nSolution: The choices of men and women are independent of each other; therefore, we apply the Product Rule to the two blocks:\n$$C(6, 3) \\times C(5, 2) = 20 \\times 10 = \\mathbf{200}$$ Sample of 3 parts: at least 1 defective # Problem: In a lot of 20 parts (5 defective, 15 good), a random sample of 3 parts is drawn. How many samples contain at least 1 defective part?\nSolution: Counting the favorable cases directly would require summing three subcases (1 defective + 2 good; 2 defective + 1 good; 3 defective). The complement strategy is more efficient:\n$$\\text{At least 1 defective} = \\text{Total} - \\text{No defective}$$$$C(20, 3) - C(15, 3) = 1{,}140 - 455 = \\mathbf{685}$$ Splitting into equal groups # Problem: In how many ways can 10 people be split into 2 groups of 5? Consider two cases: (a) unlabeled groups; (b) labeled groups (Group A and Group B).\nSolution:\n(a) Indistinguishable (unlabeled) groups: we choose who goes to the first group, and the second is automatically determined. Since the groups are equivalent to each other, each split would be counted \\(2! = 2\\) times if we did not correct for it. We divide by the number of permutations of the \\(2\\) groups:\n$$\\frac{C(10,5)}{2!} = \\frac{252}{2} = \\mathbf{126}$$(b) Labeled groups (Group A and Group B): the split \u0026ldquo;5 in Group A and 5 in Group B\u0026rdquo; is different from \u0026ldquo;5 in Group B and 5 in Group A\u0026rdquo;. It is enough to choose who goes to Group A, since Group B is the complement:\n$$C(10,5) = \\mathbf{252}$$ Summary Table # The tables below bring together the three fundamental groupings and the properties specific to combinations established in this article.\nComparison of the groupings:\nConcept Uses all \\(n\\)? Does order matter? Formula Permutation Yes Yes \\(P_n = n!\\) Arrangement No (\\(r \\leq n\\)) Yes \\(A(n,r) = \\dfrac{n!}{(n-r)!}\\) Combination No (\\(r \\leq n\\)) No \\(C(n,r) = \\dfrac{n!}{r!(n-r)!}\\) Properties of combinations:\nProperty Formula Relation to arrangement \\(A(n, r) = C(n, r) \\cdot r!\\) Symmetry \\(C(n, r) = C(n, n-r)\\) Extreme cases \\(C(n, 0) = C(n, n) = 1\\) Unit cases \\(C(n, 1) = C(n, n-1) = n\\) Exercises # The exercises below cover the main techniques of the article: determining \\(n\\), symmetry, the complement, and the Product Rule. They are ordered from simplest to most complex.\nExercise 1: Determining \\(n\\) If \\(C(n, 2) = 10\\), find \\(n\\).\nAnswer Expanding the formula: $$\\frac{n(n-1)}{2} = 10 \\implies n^2 - n - 20 = 0 \\implies (n-5)(n+4) = 0$$ Since \\(n \u0026gt; 0\\), we get \\(n = 5\\).\nExercise 2: Committee with a chair From 12 people, 3 must be chosen for a committee. In how many ways can the committee be formed if one of the members must be the chair (a distinct role)?\nAnswer The chair is a distinct position, and it matters who fills that role. We choose the chair in an ordered way (12 options) and the other 2 members without distinction from the 11 remaining: $$12 \\times C(11, 2) = 12 \\times 55 = \\mathbf{660}$$ Exercise 3: Grid paths On a 4×4 grid of points, in how many ways can we go from the bottom-left corner to the top-right corner moving only up or right?\nAnswer Every path uses exactly 3 moves up and 3 moves right, for a total of 6 moves. The path is completely determined by choosing which 3 positions (out of the 6) are the \u0026ldquo;up\u0026rdquo; move. The figure below shows one of these paths, with the moves numbered in the order they are made:\nOne of the 20 paths: up at positions 2, 4, and 5 Each choice of the 3 \u0026ldquo;up\u0026rdquo; positions gives a different path, and every path comes from one such choice. Therefore: $$C(6, 3) = \\mathbf{20} \\text{ paths}$$ Exercise 4: Points generating lines and triangles With 10 points in the plane in general position (no 3 collinear), how many distinct lines and triangles can be formed?\nAnswer Lines: each pair of points determines exactly one line: \\(C(10,2) = \\mathbf{45}\\) Triangles: each trio of points determines exactly one triangle: \\(C(10,3) = \\mathbf{120}\\) Exercise 5: Subsets of varying size How many nonempty subsets of size ≤ 3 can be formed from 7 elements?\nAnswer We add up the combinations for each size (Sum Rule): $$C(7,1) + C(7,2) + C(7,3) = 7 + 21 + 35 = \\mathbf{63}$$ Exercise 6: Teams with a restriction In a class of 6 boys and 4 girls, how many teams of 4 have at least 1 girl?\nAnswer By complement: $$C(10,4) - C(6,4) = 210 - 15 = \\mathbf{195}$$ Next steps # The combination \\(C(n, r)\\) completes the fundamental triad of classical combinatorics: together with the permutation and the arrangement, it covers the vast majority of counting problems with distinct elements.\nThe limitation of all three tools is exactly this requirement of distinctness: each element can appear at most once. But what happens when there are repetitions? How many anagrams does the word MISSISSIPPI have? How many 7-digit numbers can be formed with the digits \\(\\{1, 1, 2, 2, 2, 3, 3\\}\\)?\nThese problems lead us to permutations with repetition, the topic of the next article in the series.\n","date":"2026-07-09","externalUrl":null,"permalink":"/2026/07/combinations/","section":"Blog","summary":"Out of 10 friends, 3 go buy the popcorn. Only who goes matters, not the order of the choice, so there are 120 possible groups. We will see the combination formula, its symmetry, the complement method, and how to split a set into groups.","title":"Combinations: When Order Does Not Matter","type":"posts"},{"content":" In a race with 10 participants, in how many ways can the podium (1st, 2nd, 3rd place) be filled? Here we use only 3 of the 10 athletes, and order matters (1st ≠ 2nd). This is an arrangement (also called an r-permutation).\nWhy arrangements? # Selecting and ordering a subset of elements, without necessarily using all of them, is a core operation in computing and mathematics:\nSecurity: passwords of length \\(r\\) with no repetition, drawn from an alphabet of \\(n\\) symbols, add up to \\(A(n, r)\\) possibilities, and the arrangement sizes the space a brute-force attacker has to search. License plates and unique identifiers: alphanumeric systems with no repeated characters use the arrangement to compute how many distinct identifiers can be issued, exactly the example solved below. Scheduling: ordering \\(r\\) processes out of \\(n\\) available in a priority queue corresponds to \\(A(n, r)\\) possible sequences. Routes and graphs: the number of simple paths that go through \\(r\\) distinct vertices, in order, in a complete graph with \\(n\\) vertices is \\(A(n, r)\\), the basis of state-space search algorithms. Rankings: podiums, ordered recommendation lists, and tournament results are arrangements, because it matters who was chosen and in which position. From permutation to arrangement # The permutation, covered in the previous article, orders all \\(n\\) elements. The arrangement relaxes that requirement: we order only \\(r \u0026lt; n\\) of them. The diagram below places the arrangement in the family of groupings:\n\u0026#x2197; flowchart TD A[\"n distinct elements\"] --\u003e B{\"Use all of them?\"} B --\u003e|\"Yes\"| C[\"Permutation\nP_n = n!\"] B --\u003e|\"No: choose r \u003c n\"| D{\"Does order matter?\"} D --\u003e|\"Yes\"| E[\"Arrangement\nA(n,r) = n!/(n-r)!\"] D --\u003e|\"No\"| F[\"Combination\n(next article)\"] Arrangement # Definition An arrangement of \\(n\\) distinct elements taken \\(r\\) at a time (an r-permutation) is an ordered selection of \\(r\\) distinct elements chosen from \\(n\\) (\\(r \\leq n\\)):\n$$A(n, r) = \\frac{n!}{(n-r)!} = n \\cdot (n-1) \\cdots (n-r+1)$$In the international literature, the same number usually appears as \\(P(n, r)\\) or \\({}_{n}P_{r}\\), from permutation, and it is the nPr key on scientific calculators.\nNotable cases:\n\\(A(n, n) = n!\\), the same as the permutation \\(P_n\\) \\(A(n, 1) = n\\) Back to the opening example: the podium of a race with 10 athletes has \\(A(10, 3) = 10 \\times 9 \\times 8 = 720\\) distinct configurations.\nThe figure below shows where this product comes from. Each podium place is a little box with the number of athletes still available for it, and each filled place leaves one athlete fewer for the following ones; in the general case, the \\(r\\) boxes carry \\(n, n-1, \\ldots, n-r+1\\):\nThe podium in little boxes: 10 × 9 × 8 = 720 Worked Examples # Building with 8 floors: distinct (entry, exit) pair # Problem: A building has 8 floors. A person gets in the elevator on one floor and gets out on a different one. In how many distinct ways can this ordered (entry, exit) pair occur?\nSolution: We want to choose 2 distinct floors out of 8, where order matters, since getting in at the 3rd and out at the 7th is different from getting in at the 7th and out at the 3rd. This is an arrangement of 8 elements taken 2 at a time.\nWe fill the two positions sequentially:\nPosition Available options Justification Entry floor 8 any of the 8 floors Exit floor 7 any except the entry floor $$A(8, 2) = \\frac{8!}{(8-2)!} = \\frac{8!}{6!} = 8 \\times 7 = \\mathbf{56}$$ SQL: CROSS JOIN with a filter generates distinct ordered pairs In the article on the Sum and Product Rules, we saw that CROSS JOIN, an SQL (Structured Query Language) operation that generates the Cartesian product of tables, produces \\(n^2\\) pairs. Adding WHERE a.floor \u0026lt;\u0026gt; b.floor eliminates the \\(n\\) diagonal pairs where entry = exit and produces exactly the \\(A(n, 2)\\) ordered pairs:\nCREATE TABLE floors (floor INT); INSERT INTO floors VALUES (1),(2),(3),(4),(5),(6),(7),(8); SELECT a.floor AS entry, b.floor AS exit FROM floors a CROSS JOIN floors b WHERE a.floor \u0026lt;\u0026gt; b.floor ORDER BY entry, exit; -- 56 rows = A(8, 2) SELECT COUNT(*) FROM floors a CROSS JOIN floors b WHERE a.floor \u0026lt;\u0026gt; b.floor; -- 56 The condition WHERE a.floor \u0026lt;\u0026gt; b.floor is exactly what distinguishes the arrangement \\(A(n, 2) = n(n-1)\\) from the Cartesian product \\(n^2\\): it removes the \\(n\\) rows where the person would get in and out on the same floor.\nSchedule of 3 activities out of 4 # Problem: A student has 4 activities available: reading (R), exercise (E), drawing (D), and cooking (C). They want to put together a daily schedule with 3 of these activities in a fixed sequence: order matters, because it determines what they will do in the morning, afternoon, and evening. In how many distinct ways can they put together this schedule?\nSolution: We want to select and order 3 distinct activities out of 4. We fill the three positions sequentially, with no repetition:\nPeriod Available options Justification Morning 4 any of the 4 activities Afternoon 3 any except the morning one Evening 2 any except the two already chosen $$A(4, 3) = \\frac{4!}{(4-3)!} = \\frac{4!}{1!} = 4 \\times 3 \\times 2 = \\mathbf{24}$$ Routes among 5 cities: ordered route of 3 cities # Problem: A traveler needs to plan a route that goes through exactly 3 cities among São Paulo, Rio de Janeiro, Belo Horizonte, Porto Alegre, and Curitiba. The order of the cities visited determines the route: SP → RJ → BH is different from RJ → SP → BH. How many distinct routes can they plan?\nSolution: We want an ordered sequence of 3 distinct cities chosen from 5. We fill the three stages of the route sequentially:\nRoute stage Available options Justification 1st city 5 any of the 5 cities 2nd city 4 any except the 1st 3rd city 3 any except the previous two $$A(5, 3) = 5 \\times 4 \\times 3 = \\mathbf{60}$$ Python: itertools.permutations and the arrangement formula itertools.permutations(iterable, r) generates exactly the \\(A(n, r)\\) arrangements of \\(r\\) distinct elements drawn from a sequence of \\(n\\) unique elements (the function distinguishes them by position, not by value):\nimport itertools, math # A(5, 3) — routes of 3 cities among 5 cities = [\u0026#34;SP\u0026#34;, \u0026#34;RJ\u0026#34;, \u0026#34;BH\u0026#34;, \u0026#34;POA\u0026#34;, \u0026#34;CWB\u0026#34;] routes = list(itertools.permutations(cities, 3)) print(len(routes)) # 60 # Confirm with the formula def arrangement(n, r): return math.factorial(n) // math.factorial(n - r) print(arrangement(5, 3)) # 60 For large \\(n\\) and \\(r\\), never materialize the iterator as a list; prefer counting with the formula. \\(A(26, 3) \\times A(10, 4) = 78{,}624{,}000\\) objects would demand prohibitive memory.\nSQL: triple CROSS JOIN for routes of 3 cities Extending the pattern from the article on the Sum and Product Rules: three chained CROSS JOINs with distinctness filters generate \\(A(n, 3)\\) ordered routes without revisiting cities.\nCREATE TABLE cities (city TEXT); INSERT INTO cities VALUES (\u0026#39;SP\u0026#39;),(\u0026#39;RJ\u0026#39;),(\u0026#39;BH\u0026#39;),(\u0026#39;POA\u0026#39;),(\u0026#39;CWB\u0026#39;); SELECT a.city AS origin, b.city AS stopover, c.city AS destination FROM cities a CROSS JOIN cities b CROSS JOIN cities c WHERE a.city \u0026lt;\u0026gt; b.city AND b.city \u0026lt;\u0026gt; c.city AND a.city \u0026lt;\u0026gt; c.city ORDER BY origin, stopover, destination; SELECT COUNT(*) FROM cities a CROSS JOIN cities b CROSS JOIN cities c WHERE a.city \u0026lt;\u0026gt; b.city AND b.city \u0026lt;\u0026gt; c.city AND a.city \u0026lt;\u0026gt; c.city; -- 60 = A(5, 3) The pattern generalizes to any \\(r\\): we chain \\(r\\) copies of the table with CROSS JOIN and filter all pairs of aliases with \u0026lt;\u0026gt;. For large \\(r\\), this approach becomes impractical in SQL, so use the formula.\nLicense plates: 3 letters and 4 distinct digits # Problem: A plate system uses the format AAA-NNNN, with 3 letters of the alphabet (26 letters) followed by 4 digits (from 0 to 9). No letter may repeat in the 3 letter positions, and no digit may repeat in the 4 digit positions. How many distinct plates can be issued?\nSolution: Choosing the letters and choosing the digits are independent steps, so we apply the Product Rule between the two blocks.\nLetter block, \\(A(26, 3)\\): we select and order 3 distinct letters out of 26.\nPosition Available options Justification 1st letter 26 any of the 26 letters 2nd letter 25 any except the 1st 3rd letter 24 any except the previous two $$A(26, 3) = 26 \\times 25 \\times 24 = 15{,}600$$Digit block, \\(A(10, 4)\\): we select and order 4 distinct digits out of 10.\nPosition Available options Justification 1st digit 10 any of the digits from 0 to 9 2nd digit 9 any except the 1st 3rd digit 8 any except the previous two 4th digit 7 any except the previous three $$A(10, 4) = 10 \\times 9 \\times 8 \\times 7 = 5{,}040$$Total, by the Product Rule between the two blocks:\n$$A(26, 3) \\times A(10, 4) = 15{,}600 \\times 5{,}040 = \\mathbf{78{,}624{,}000}$$ 3-digit numbers with distinct digits from \\(\\{0, 1, \\ldots, 9\\}\\) # Problem: How many 3-digit numbers (between 100 and 999), with all digits distinct, can be formed with the digits \\(\\{0, 1, \\ldots, 9\\}\\)?\nSolution: The difficulty is the restriction: the 1st digit cannot be 0, otherwise the number would effectively have only 2 digits (e.g., 053 = 53). We present two approaches.\nDirect approach (fill the restricted position first):\nThe strategy is always to start with the position that has a restriction, then fill the others.\nPosition Available options Justification Hundreds (1st) 9 digits from 1 to 9 (0 excluded) Tens (2nd) 9 digits from 0 to 9 except the hundreds digit (0 is an option again) Units (3rd) 8 digits from 0 to 9 except the two already used $$9 \\times 9 \\times 8 = \\mathbf{648}$$Check by complement:\nWe count all arrangements of 3 distinct digits (including those that start with 0) and subtract the invalid ones:\nTotal without restriction: \\(A(10, 3) = 10 \\times 9 \\times 8 = 720\\) Start with 0: 0 fixed in the 1st position, 2 digits chosen from the remaining 9 → \\(A(9, 2) = 9 \\times 8 = 72\\) $$A(10,3) - A(9,2) = 720 - 72 = \\mathbf{648} \\checkmark$$ Python: generating and checking PINs with no repeated digits This example is the classic problem of sizing the space of PINs (Personal Identification Numbers) with no repetition, a context introduced in the section Why arrangements?. The code below enumerates the PINs, applies the validity filter, and confirms the complement method:\nfrom itertools import permutations digits = range(10) # {0, 1, ..., 9} # All arrangements of 3 distinct digits (including those starting with 0) all_pins = list(permutations(digits, 3)) print(len(all_pins)) # 720 = A(10, 3) # Only those valid as a 3-digit number (no 0 in the 1st position) valid = [p for p in all_pins if p[0] != 0] print(len(valid)) # 648 # Invalid cases: start with 0 → A(9, 2) arrangements of the 9 remaining digits invalid = [p for p in all_pins if p[0] == 0] print(len(invalid)) # 72 = A(9, 2) # Confirm the complement assert len(valid) + len(invalid) == len(all_pins) # 648 + 72 = 720 ✓ The last line confirms in code what we did on paper: \\(A(10, 3) - A(9, 2) = 720 - 72 = 648\\).\nEven 4-digit numbers with distinct digits from \\(\\{0,1,\\ldots,9\\}\\) # Problem: How many 4-digit numbers (between 1000 and 9999), with all digits distinct, are even, using the digits \\(\\{0, 1, \\ldots, 9\\}\\)?\nSolution: A number is even when its last digit (units) belongs to \\(\\{0, 2, 4, 6, 8\\}\\). We split into two cases according to the final digit, because the restriction on the 1st digit (≠ 0) changes depending on which even digit occupies the last position.\nCase I (last digit = 0):\nWith 0 fixed in the last position, it cannot appear in the 1st, so the restriction on the 1st digit is already satisfied.\nPosition Available options Justification Units (4th) 1 fixed at 0 Thousands (1st) 9 digits from 1 to 9 (0 already used) Hundreds (2nd) 8 remaining digits except the two already used Tens (3rd) 7 remaining digits $$M_1 = 9 \\times 8 \\times 7 = \\mathbf{504}$$Case II (last digit \\(\\in \\{2, 4, 6, 8\\}\\)):\nNow 0 was not used in the last position, so it remains forbidden in the 1st position.\nPosition Available options Justification Units (4th) 4 choice from \\(\\{2, 4, 6, 8\\}\\) Thousands (1st) 8 digits from 1 to 9 except the one already used in the units Hundreds (2nd) 8 digits from 0 to 9 except the two already used (0 is an option again) Tens (3rd) 7 remaining digits $$M_2 = 4 \\times 8 \\times 8 \\times 7 = \\mathbf{1{,}792}$$Total, by the Sum Rule between the two cases:\n$$M_1 + M_2 = 504 + 1{,}792 = \\mathbf{2{,}296}$$ Summary Table: Permutation vs. Arrangement # Concept Uses all? Order matters? Formula Permutation Yes Yes \\(n!\\) Arrangement No (\\(r \u0026lt; n\\)) Yes \\(\\dfrac{n!}{(n-r)!}\\) Note In general, it is convenient to start the analysis with the positions that have special restrictions (e.g., it cannot be 0 in the first position).\nExercises # Exercise 1: Coordinator and deputy coordinator In a committee of 10 professors, in how many ways can a coordinator and a deputy coordinator be chosen?\nAnswer Order matters (coordinator ≠ deputy coordinator): $$A(10,2) = 10 \\times 9 = \\mathbf{90}$$ Exercise 2: Finding \\(n\\) (a) If \\(A(n,2) = 72\\), find \\(n\\). (b) If \\(4A(n,2) = A(2n,3)\\), is there a solution?\nSolution (a) \\(n(n-1) = 72 \\Rightarrow n^2 - n - 72 = 0 \\Rightarrow \\mathbf{n = 9}\\)\n(b) \\(4n(n-1) = 2n(2n-1)(2n-2) = 4n(n-1)(2n-1)\\). Dividing both sides by \\(4n(n-1)\\) (valid for \\(n \\geq 2\\)): \\(1 = 2n-1 \\Rightarrow n = 1\\). But this contradicts \\(n \\geq 2\\). There is no solution.\nExercise 3: Photo of 4 friends out of 10 In how many ways can 4 friends out of 10 line up for a photo?\nAnswer Order (position in the photo) matters: $$A(10,4) = 10 \\times 9 \\times 8 \\times 7 = \\mathbf{5{,}040}$$ Exercise 4: Airline tickets An airline connects 7 cities. How many types of tickets (origin → destination) exist?\nAnswer \\(A(7,2) = 7 \\times 6 = \\mathbf{42}\\)\nExercise 5: 3-digit numbers from \\(\\{2,3,5,8,9\\}\\) With the digits \\(\\{2,3,5,8,9\\}\\), count the 3-digit numbers with distinct digits that are: (a) multiples of 5 (b) even (c) odd (d) less than 800\nSolution (a) Units = 5: \\(A(4,2) = \\mathbf{12}\\)\n(b) Units \\(\\in \\{2,8\\}\\): \\(2 \\times A(4,2) = \\mathbf{24}\\)\n(c) Units \\(\\in \\{3,5,9\\}\\): \\(3 \\times A(4,2) = \\mathbf{36}\\)\n(d) Hundreds \\(\\in \\{2,3,5\\}\\): \\(3 \\times A(4,2) = \\mathbf{36}\\)\nNext steps # The arrangement solves the case where order matters and we use only \\(r \u0026lt; n\\) elements. Its limitation is exactly that requirement of order: in many practical situations (forming a committee, choosing the ingredients of a recipe, selecting movies to watch) all that matters is which elements were chosen, not in what sequence.\nWhen order does not matter, we arrive at combinations, the next topic in the series. With permutations, arrangements, and combinations we will have the fundamental triad of combinatorics, enough to solve the vast majority of counting problems.\n","date":"2026-07-07","externalUrl":null,"permalink":"/2026/07/arrangements/","section":"Blog","summary":"In a race with 10 athletes, the podium can be filled in 720 ways. That is an arrangement, an ordered selection of some elements of a set. We solve problems about routes, license plates, and numbers with distinct digits, with Python and SQL.","title":"Arrangements: Ordering Without Using Everything","type":"posts"},{"content":" How many anagrams does the word VIRUS have? In how many ways can 5 friends sit around a round table? These questions have a different structure from the ones we saw in the article on the Sum and Product Rules: here we want to order all the elements of a set, not just choose some of them.\nWhy permutations? # Ordering elements is a fundamental operation in computing and mathematics:\nAlgorithm analysis: to sort \\(n\\) elements, there are \\(n!\\) possible arrangements. The contrast between an O(n!) algorithm, which would test every order, and an O(n log n) one such as mergesort shows why choosing the right algorithm can mean the difference between milliseconds and astronomical time. Cryptography: transposition ciphers rearrange the characters of a text as an encoding operation. Security depends, in part, on how many permutations exist: at most \\(n!\\) for a text of \\(n\\) distinct characters, although practical ciphers use only a fraction of them. Task scheduling: \\(n\\) independent processes can be run in \\(n!\\) orders. Finding the lowest-cost sequence is a permutation problem: for large \\(n\\), brute force is impractical. Integration testing: covering every execution order of \\(n\\) concurrent operations requires testing \\(n!\\) sequences. Formal verification tools (model checking) explore the possible execution orders to verify properties of concurrent systems, such as the absence of race conditions. Routing: the number of routes that leave a city, visit the others exactly once, and return to the origin is \\((n-1)!\\), the basis of the classic Traveling Salesman Problem (TSP). This article formalizes the simple permutation (ordering in a line) and the circular permutation (ordering in a circle), with examples of anagrams, round tables, and adjacency restrictions.\nFactorial # Before permutations themselves, we need the factorial:\nFactorial $$n! = n \\cdot (n-1) \\cdot (n-2) \\cdots 2 \\cdot 1, \\qquad 0! = 1$$ \\(n\\) \\(n!\\) 0 1 1 1 2 2 3 6 4 24 5 120 6 720 Why \\(0! = 1\\)? It is not an arbitrary convention. First, the relation \\(n! = n \\cdot (n-1)!\\) only holds for \\(n = 1\\) if \\(1! = 1 \\cdot 0!\\), which forces \\(0! = 1\\). Second, \\(n!\\) counts the ways of ordering \\(n\\) objects, and there is exactly one way of ordering zero objects: doing nothing.\nIntuition: if we have 4 pens of different colors to hand out among 4 friends, the first receives any of the 4, the second any of the 3 remaining, and so on: \\(4 \\times 3 \\times 2 \\times 1 = 4! = 24\\) distributions.\nSimple Permutation # Definition A simple permutation of \\(n\\) distinct elements is an ordering of all those elements in a line. $$P_n = n!$$Justification: \\(n\\) choices for the 1st element, \\(n-1\\) for the 2nd, \u0026hellip;, 1 for the last: \\(n \\cdot (n-1) \\cdots 1 = n!\\)\nExamples # Example 1: Anagrams of VIRUS\nStatement: In how many distinct ways can the 5 letters of the word VIRUS be arranged?\nSolution: The 5 letters are all distinct and we want to order all of them in a line, which characterizes a simple permutation. We count the choices position by position:\nPosition Available options Reason 1st 5 any of the 5 letters 2nd 4 one letter already used 3rd 3 two letters already used 4th 2 three letters already used 5th 1 only one letter left $$P_5 = 5! = 5 \\times 4 \\times 3 \\times 2 \\times 1 = 120$$Example 2: Anagrams of VIRUS that start and end with a consonant\nStatement: How many anagrams of VIRUS start and end with a consonant?\nSolution: VIRUS has 3 consonants (V, R, S) and 2 vowels (I, U).\nThe strategy is to fill the restricted positions first (1st and 5th), then the others:\nPosition Options Reason 1st (must be a consonant) 3 any of the 3 consonants 5th (must be a consonant) 2 from the 2 remaining consonants 2nd, 3rd, 4th (free) 3! = 6 the 3 letters left over (2 vowels + 1 consonant) in any order $$3 \\times 2 \\times 3! = 3 \\times 2 \\times 6 = 36$$Filling the restricted positions first avoids the mistake of computing the total and then trying to \u0026ldquo;remove\u0026rdquo; invalid cases: here the Product Rule already guarantees that every arrangement counted is valid.\nPython: itertools.permutations and math.factorial Python\u0026rsquo;s standard library offers two complementary paths: enumerating the permutations or just counting how many there are.\nimport itertools import math letters = \u0026#34;VIRUS\u0026#34; # Count: P_5 = 5! = 120 print(math.factorial(len(letters))) # 120 # Enumerate and filter: starting and ending with a consonant consonants = set(\u0026#34;VRS\u0026#34;) anagrams = itertools.permutations(letters) filtered = [a for a in anagrams if a[0] in consonants and a[-1] in consonants] print(len(filtered)) # 36 itertools.permutations(iterable, r) generates all permutations of length \\(r\\) as tuples (omitting \\(r\\) uses all the elements). For large problems, never convert the iterator to a list: \\(20! \u0026gt; 2 \\times 10^{18}\\).\nExample 3: 5-digit numbers with \\(\\{0, 5, 7, 8, 9\\}\\)\nStatement: Using each digit of the set \\(\\{0, 5, 7, 8, 9\\}\\) exactly once, how many 5-digit numbers can be formed?\nSolution: The set has 5 distinct digits, and without restrictions there would be \\(P_5 = 120\\) arrangements. The restriction is that the digit 0 cannot occupy the 1st position: a number with 0 in front would effectively have only 4 digits.\nDirect reasoning: fill the restricted position first:\nPosition Options Reason 1st 4 \\(\\{5, 7, 8, 9\\}\\), with 0 excluded 2nd 4 the 4 remaining digits, including 0 3rd 3 4th 2 5th 1 $$4 \\times 4! = 4 \\times 24 = 96$$Check by complement: subtract the invalid cases from the total:\n$$5! - 4! = 120 - 24 = 96 \\checkmark$$The \\(4!\\) cases removed correspond to all the arrangements in which 0 occupies the 1st position and the 4 remaining digits fill the other positions freely.\nExample 4: Concert: 4 women and 5 men in consecutive seats, groups kept apart\nStatement: Nine friends attend a concert with assigned, consecutive seats. The 4 women must all sit together, and so must the 5 men. In how many ways can they sit?\nSolution: Since the seats are assigned and consecutive (a linear row), there are two possibilities for the order of the blocks: [W W W W | M M M M M] or [M M M M M | W W W W]. Within each block, the members order themselves freely.\nStep Calculation Reason Relative position of the blocks 2 women\u0026rsquo;s block to the left or to the right of the men\u0026rsquo;s Internal permutations of the women \\(4! = 24\\) the 4 women order themselves within the block Internal permutations of the men \\(5! = 120\\) the 5 men order themselves within the block $$2 \\times 4! \\times 5! = 2 \\times 24 \\times 120 = \\mathbf{5{,}760}$$ Circular Permutation # When arranging people around a round table, what matters is the relative position: there is no \u0026ldquo;first chair\u0026rdquo;. Rotations of the same arrangement count as the same permutation.\nCircular Permutation The number of circular permutations of \\(n\\) distinct elements is: $$PC_n = \\frac{P_n}{n} = (n-1)!$$Intuition: each circular permutation corresponds to \\(n\\) linear permutations (each rotation produces a new ordering in a line, but the same arrangement in a circle). Therefore, we divide by \\(n\\).\nThe figure below shows this for 5 people. Each table is the previous one rotated by one chair; read from the same chair, the five give different lines, but in all of them each person has the same neighbors:\nThe 5 rotations are the same arrangement at the table Examples # Example 1: 5 friends around a round table\nStatement: In how many ways can 5 friends sit around a round table?\nSolution: In a line, each person\u0026rsquo;s position is absolute. At a round table, what matters is the relative position among those present: shifting everyone one chair to the right does not create a new arrangement, just a rotation of the same one.\nTo eliminate equivalent rotations, we fix one person in any chair and permute the other 4:\n$$PC_5 = (5-1)! = 4! = 24$$ Application: the Traveling Salesman Problem (TSP) The problem: a salesman needs to visit \\(n\\) cities exactly once and return to the city of origin. Between each pair of cities there is a known distance. What is the route with the smallest total distance?\nFormally: given a complete graph with \\(n\\) vertices (cities) and weighted edges (distances), find the lowest-cost Hamiltonian cycle, the one that passes through every vertex exactly once.\nWhy it belongs here: the route forms a cycle and the starting city does not change the structure of the tour, which is exactly the definition of a circular permutation. The number of distinct routes to evaluate is \\((n-1)!\\), or \\((n-1)!/2\\) when the distance from one city to another equals the distance back, because each route and the same route traveled in the opposite direction have the same length.\nThe scale of the problem: for \\(n = 20\\) cities, there are \\(19! \\approx 1.2 \\times 10^{17}\\) possible routes. If a computer evaluated \\(10^9\\) routes per second, it would take almost 4 years to test them all. For \\(n = 30\\), the time would exceed the age of the universe.\nWhat NP-hard means: the class NP gathers the problems whose solutions can be verified in polynomial time. A problem is NP-hard when every problem in NP can be reduced to it in polynomial time, that is, it is at least as hard as any of them. Therefore, a polynomial algorithm for the TSP would solve all the problems in NP at once. Nobody knows such an algorithm, and nobody has proved that it does not exist: this is the question P versus NP, one of the great open problems of mathematics. The best known exact algorithm for the TSP has exponential complexity. In practice, this means that for large instances the problem is intractable by brute force, and one resorts to approximation algorithms (which guarantee a solution within a factor of the optimum) or heuristics such as genetic algorithms and simulated annealing (which find good solutions with no guarantee of optimality). The TSP is one of the most studied problems in combinatorial optimization.\nExamples 2 and 3: Two non-adjacent elements: complement vs. sequential insertion\nThe next two examples solve the same type of problem (two specific elements cannot be adjacent in a circular arrangement) with different strategies. The goal is to show when each approach is more natural.\nThe diagram below summarizes the choice: the key question is whether the forbidden cases are easy to count directly.\n\u0026#x2197; flowchart TD A[\"Two elements cannot be adjacent\nin a circular arrangement\"] --\u003e B{\"Are the forbidden cases\neasy to count\nas a block?\"} B --\u003e|Yes| C[\"Complement\nTotal − Forbidden\"] B --\u003e|No| D[\"Sequential insertion\nBuild avoiding\nthe restriction from the start\"] C --\u003e E[\"Ex. 2: 5 children\n24 − 12 = 12\"] D --\u003e F[\"Ex. 3: 6 children\n6 × 4 × 3 = 72\"] Example 2: Ring dance of 5 children: the complement strategy\nStatement: Five children (\\(c_1, c_2, c_3, c_4, c_5\\)) form a ring dance circle. In how many ways can they be arranged so that \\(c_1\\) and \\(c_2\\) are not side by side?\nStrategy (complement): instead of directly counting the valid cases (hard), we count the total and subtract the forbidden cases (easier).\n$$\\text{valid} = \\text{total} - \\text{cases with } c_1 \\text{ and } c_2 \\text{ together}$$Total without restriction:\n$$PC_5 = (5-1)! = 4! = 24$$Forbidden cases (\\(c_1\\) and \\(c_2\\) together): we treat \\(\\{c_1, c_2\\}\\) as a single block. The problem becomes arranging 4 entities in a circle and ordering the block internally:\nStep Calculation Reason 4 entities in a circle \\((4-1)! = 3! = 6\\) block + \\(c_3, c_4, c_5\\) Ordering \\(c_1\\) and \\(c_2\\) within the block \\(2! = 2\\) \\(c_1 c_2\\) or \\(c_2 c_1\\) Forbidden cases: \\(6 \\times 2 = 12\\)\n$$PC_5 - \\text{forbidden} = 24 - 12 = \\mathbf{12}$$When to use the complement: when the forbidden cases form a compact set that is easy to describe. Here, it is enough to treat the pair as a block.\nExample 3: Circle of 6 children: the sequential insertion strategy\nStatement: Six children (\\(c_1\\) to \\(c_6\\)) form a ring dance circle. In how many ways can they be arranged so that \\(c_1\\) and \\(c_2\\) are not side by side?\nStrategy (sequential insertion): we first place the unrestricted elements, creating a structure of gaps, and insert the problematic elements last, choosing only valid positions from the start.\nStep Calculation Reason Arrange \\(c_3, c_4, c_5, c_6\\) in a circle \\((4-1)! = 3! = 6\\) 4 unrestricted children form the base structure Insert \\(c_1\\) into one of the 4 gaps between them \\(6 \\times 4 = 24\\) 4 gaps available: any position is valid for \\(c_1\\) Insert \\(c_2\\) in a position not adjacent to \\(c_1\\) \\(24 \\times 3 = 72\\) of the 5 remaining gaps, 2 are adjacent to \\(c_1\\); 3 valid ones remain $$\\mathbf{72} \\text{ arrangements}$$The diagram below illustrates the three stages of the construction. Each arrow represents a multiplication by the number of choices available at that step:\n\u0026#x2197; flowchart LR A[\"Stage 1\nc₃ c₄ c₅ c₆\nin a circle\n(PC)₄ = 6\"] --\u003e|\"× 4 positions\nfor c₁\"| B[\"Stage 2\nInsert c₁\n24 arrangements\"] --\u003e|\"× 3 non-adjacent\npositions\nfor c₂\"| C[\"Stage 3\nInsert c₂\n72 arrangements\"] When to use sequential insertion: when the complement would be hard to compute. For example, if there were multiple forbidden pairs, counting by complement would require inclusion-exclusion. Sequential insertion builds the restriction directly into the construction, with nothing to subtract.\nExample 4: Concert revisited: what if the stage were circular?\nStatement: The same 9 friends from Example 4 of the previous section, 4 women and 5 men, now position themselves around a circular stage, keeping the restriction that all the women stay together and all the men together. In how many ways can they be arranged?\nSolution: The block structure is the same, but the context changed from linear to circular. What changes?\nIn the linear case, the 2 blocks could occupy two distinct orders in the line: [W|M] or [M|W]. In the circle, rotating everyone one position turns [W|M] into [M|W]: the two orders are rotations of each other and count as the same arrangement. With only 2 blocks in a circle, there is \\((2-1)! = 1\\) circular arrangement.\nStep Linear (previous Ex. 4) Circular (this example) Arrangement of the blocks 2 (order matters in a line) \\((2-1)! = 1\\) (equivalent rotations) Internal permutations of the women \\(4! = 24\\) \\(4! = 24\\) Internal permutations of the men \\(5! = 120\\) \\(5! = 120\\) Total \\(2 \\times 24 \\times 120 = \\mathbf{5{,}760}\\) \\(1 \\times 24 \\times 120 = \\mathbf{2{,}880}\\) The factor 2 that existed in the linear case disappears in the circular one: in a circle, \u0026ldquo;women to the left of the men\u0026rdquo; and \u0026ldquo;men to the left of the women\u0026rdquo; describe the same relative arrangement, just rotated.\nPython: simulating circular permutations To generate circular permutations without counting rotations as distinct, just fix one element and permute the others:\nimport itertools import math people = [\u0026#34;Ana\u0026#34;, \u0026#34;Luís\u0026#34;, \u0026#34;Fernando\u0026#34;, \u0026#34;Clara\u0026#34;, \u0026#34;Bruno\u0026#34;] # Fix \u0026#34;Ana\u0026#34; and permute the other 4: (5-1)! = 4! = 24 fixed = people[0] others = people[1:] circular = list(itertools.permutations(others)) print(len(circular)) # 24 print(math.factorial(len(people) - 1)) # 24 Fixing one element is exactly what the formula \\((n-1)!\\) formalizes: it removes the \\(n\\) rotationally equivalent arrangements, leaving only the structurally distinct ones.\nComparing Linear and Circular Permutations # For 3 people (Ana, Luís, Fernando):\nArrangement Counts as Total Line (numbered positions) Simple permutation \\(3! = 6\\) Triangle with numbered seats Simple permutation \\(3! = 6\\) Round table (only relative position matters) Circular permutation \\((3-1)! = 2\\) Derangements: Nobody in Their Own Place # One last family of restricted permutations has a name of its own. In a gift exchange, \\(n\\) people put their gifts in a box and each draws one. In how many ways can the draw come out with nobody getting their own gift? The problem was posed by Pierre Rémond de Montmort in 1708, in the Essay d\u0026rsquo;analyse sur les jeux de hazard, and solved by him and by Nicolaus Bernoulli around 1713 (see the entry Derangement).\nDerangement A derangement of \\(n\\) elements is a permutation in which no element occupies its original position. The number of derangements is denoted \\(D_n\\).\nSmall cases. With the letters A, B, C in positions 1, 2, 3, there are only two derangements: BCA and CAB, so \\(D_3 = 2\\). With A, B, C, D there are nine, \\(D_4 = 9\\):\nBADC · BCDA · BDAC · CADB · CDAB · CDBA · DABC · DCAB · DCBA\nAnd, in the trivial cases, \\(D_1 = 0\\) (a single element has nowhere to go) and \\(D_2 = 1\\) (BA).\nA recurrence by counting. In a derangement, element 1 goes to some position \\(k \\neq 1\\): there are \\(n-1\\) choices, and each one yields the same number of derangements. With position \\(k\\) fixed, we look at where element \\(k\\) goes:\nElement \\(k\\) goes to position 1 (the two swap places): the other \\(n-2\\) elements must form a derangement among themselves, which gives \\(D_{n-2}\\) ways. Element \\(k\\) does not go to position 1: there remain \\(n-1\\) elements (all but for \\(n-1\\) positions (all but \\(k\\)), each element with one forbidden position (its own and, for element \\(k\\), position 1). This is exactly a derangement of \\(n-1\\) elements: \\(D_{n-1}\\) ways. Adding the two cases and multiplying by the \\(n-1\\) choices of \\(k\\):\n$$D_n = (n-1)\\left(D_{n-1} + D_{n-2}\\right), \\qquad D_1 = 0,\\ D_2 = 1$$Checking: \\(D_3 = 2(1 + 0) = 2\\) and \\(D_4 = 3(2 + 1) = 9\\), as in the list above; and \\(D_5 = 44\\) and \\(D_6 = 265\\) follow.\nThe closed-form formula and the number \\(e\\). The Inclusion-Exclusion Principle leads to the formula \\(D_n = n! \\sum_{k=0}^{n} (-1)^k/k!\\), derived in the article on cardinality. From it comes the most surprising result of the topic: the fraction of draws in which nobody gets their own gift, \\(D_n/n!\\), approaches \\(1/e\\), about 36.8%, practically the same for 10 people or for 10 thousand.\nPython: checking the recurrence by brute force For small \\(n\\), we can enumerate all the permutations and count the ones that leave nobody in their place:\nfrom itertools import permutations def brute_force_derangements(n): return sum(1 for p in permutations(range(n)) if all(p[i] != i for i in range(n))) D = {1: 0, 2: 1} for n in range(3, 9): D[n] = (n - 1) * (D[n - 1] + D[n - 2]) print([D[n] for n in range(1, 9)]) # [0, 1, 2, 9, 44, 265, 1854, 14833] print(all(D[n] == brute_force_derangements(n) for n in range(1, 9))) # True Summary Table # Type Formula When to use Simple permutation \\(P_n = n!\\) Line: absolute position matters Circular permutation \\(PC_n = (n-1)!\\) Circle: relative position matters With a required block \\(k! \\times P_{\\text{remaining}}\\) Elements that must stay together With a restriction (e.g., 0 at the start) Complement Remove invalid cases Derangement \\(D_n = (n-1)(D_{n-1} + D_{n-2})\\) Nobody in the original position Exercises # Exercise 1: Simplifying factorials Simplify: (a) \\(\\dfrac{(n+1)!}{n!}\\) (b) \\(\\dfrac{n!}{(n+2)!}\\) (c) \\(\\dfrac{(n+1)!}{(n-1)!}\\)\nAnswers (a) \\(n+1\\)\n(b) \\(\\dfrac{1}{(n+2)(n+1)}\\)\n(c) \\((n+1) \\cdot n\\)\nExercise 2: Anagrams of ÂNGULO How many anagrams of the word ÂNGULO (Portuguese for \u0026ldquo;angle\u0026rdquo;) (a) start with a vowel? (b) start and end with a vowel? (c) do not have the letters A and N together?\nSolution ÂNGULO has 6 distinct letters: vowels Â, U, O; consonants N, G, L. Total: \\(6! = 720\\).\n(a) 3 choices for the 1st position × \\(5! = 120\\): \\(3 \\times 120 = \\mathbf{360}\\)\n(b) 3 × 2 for the 1st and last positions × \\(4! = 24\\): \\(3 \\times 2 \\times 24 = \\mathbf{144}\\)\n(c) By complement: treat \\(\\{A, N\\}\\) as a block → 5 units, \\(5!\\) arrangements × 2 internal orders = \\(240\\). Not together: \\(720 - 240 = \\mathbf{480}\\)\nExercise 3: Numbered faces of a cube A cube has its faces painted in different colors. In how many ways can the numbers 1 to 6 be engraved, one on each face?\nAnswer Since each face is painted a distinct color, the 6 faces are distinguishable positions, with no ambiguity about which is which. Engraving the numbers 1 to 6, one on each face, is a permutation of 6 elements in 6 distinct positions: $$P_6 = 6! = \\mathbf{720} \\text{ ways}$$ Exercise 4: Ring dance with alternating sexes In how many ways can 5 girls and 5 boys form a ring dance circle so that people of the same sex are not next to each other?\nAnswer The sexes must alternate (B-G-B-G\u0026hellip;).\nArrange the 5 boys in a circle: \\((5-1)! = 4! = 24\\) Arrange the 5 girls in the 5 alternating places: \\(5! = 120\\) $$4! \\times 5! = 24 \\times 120 = \\mathbf{2{,}880}$$ Exercise 5: Couples in a circle with alternating sexes In how many ways can 4 couples form a ring dance circle with men and women alternating, with each couple adjacent?\nAnswer Arrange the 4 couples as units in a circle: \\((4-1)! = 3! = 6\\) arrangements. To keep the sexes alternating, all the couples must have the same internal orientation: if couple 1 is arranged as M-W (clockwise), all the others must follow the same order, since any couple with reversed orientation would put two elements of the same sex in adjacent positions. The couples\u0026rsquo; orientations are not independent: there are only 2 global choices (all M-W or all W-M). $$3! \\times 2 = 6 \\times 2 = \\mathbf{12} \\text{ ways}$$ Next steps # Permutation orders all \\(n\\) elements. When we want to select and order only \\(r \\leq n\\) of them, without necessarily using all of them, we enter the domain of arrangements (or r-permutations, also called partial permutations), the topic of the next article in the series.\nLater on, we will see how to count groupings in which order does not matter, the combinations, completing the fundamental triad of combinatorics: permutations, arrangements, and combinations.\n","date":"2026-07-02","externalUrl":null,"permalink":"/2026/07/permutations/","section":"Blog","summary":"How many anagrams does the word VIRUS have? In how many ways can 5 friends sit around a table? The answers are 120 and 24, by simple and circular permutations. We will also see adjacency restrictions, the traveling salesman problem, and derangements.","title":"Simple and Circular Permutations","type":"posts"},{"content":"","date":"2026-06-30","externalUrl":null,"permalink":"/tags/contagem/","section":"tags","summary":"","title":"contagem","type":"tags"},{"content":"","date":"2026-06-30","externalUrl":null,"permalink":"/tags/counting/","section":"tags","summary":"","title":"counting","type":"tags"},{"content":" Counting seems simple until the number of cases explodes. How many 8-character passwords exist? How many possible answer keys does a 20-question multiple-choice exam have? Listing them one by one is impractical. We need principles.\nTwo of them underlie all of combinatorics: the Sum Rule (SR) and the Product Rule (PR).\nWhy SR and PR? # Counting is a central operation in computing and engineering:\nSecurity and cryptography: the size of the space of passwords or keys determines the resistance to brute-force attacks. Quantifying that space is always an application of the PR. Software testing: techniques such as pairwise testing use SR and PR to choose a reduced set of cases that covers all combinations of pairs of parameters. Data structures: the number of indices of a multidimensional array \\(A[n_1][n_2]\\cdots[n_k]\\) is \\(n_1 \\cdot n_2 \\cdots n_k\\), a direct Cartesian product. Network protocols: the IPv4 (\\(2^{32}\\)) and IPv6 (\\(2^{128}\\)) address spaces come from the PR applied to sequences of bits. Algorithm analysis: estimating the complexity of exhaustive algorithms, the probability of collisions in hash tables, and the sizing of search spaces in AI (Artificial Intelligence) all rely on counting with SR and PR. This article develops the two principles formally, applies them in graded examples, and shows how to recognize which one to use in each situation.\nThe Intuition: \u0026ldquo;or\u0026rdquo; Is a Sum, \u0026ldquo;and\u0026rdquo; Is a Product # Consider a store with 4 math books and 3 Portuguese books.\nIn how many ways can we choose one book (a math book or a Portuguese book)? \\(4 + 3 = 7\\) In how many ways can we choose one of each (a math book and a Portuguese book)? \\(4 \\times 3 = 12\\) This intuition is the core of the two principles. The diagram below turns it into a decision guide that can be applied to any counting problem:\n\u0026#x2197; flowchart TD A[\"How to count?\"] --\u003e B{\"Mutually\nexclusive\nalternatives?\"} B --\u003e|\"Yes: 'or'\"| C[\"Sum Rule\n|A₁| + |A₂| + …\"] B --\u003e|\"No: 'and'\"| D{\"Independent\nstages?\"} D --\u003e|Yes| E[\"Product Rule\n|A₁| × |A₂| × …\"] D --\u003e|No| F[\"Complement\nor PIE\"] C --\u003e G[\"⚠️ Non-disjoint\nsets? → PIE\"] Sum Rule # Sum Rule (SR) If \\(A \\cap B = \\emptyset\\), then \\(|A \\cup B| = |A| + |B|\\).\nExtension: for pairwise disjoint sets \\(A_1, A_2, \\ldots, A_n\\): $$|A_1 \\cup \\cdots \\cup A_n| = |A_1| + \\cdots + |A_n|$$Application: when the choices are mutually exclusive, that is, they cannot occur simultaneously.\nWhat if the sets are not disjoint? The SR requires pairwise disjoint sets. If there are common elements, naively adding the sizes counts those elements more than once. The fix is the Inclusion-Exclusion Principle (PIE):\n$$|A \\cup B| = |A| + |B| - |A \\cap B|$$Example: in a class, 18 students play soccer and 12 play volleyball; 5 play both sports. The total number of players is \\(18 + 12 - 5 = 25\\), not \\(18 + 12 = 30\\). The PIE is covered in detail in the article on cardinality.\nCartesian Product and Product Rule # Cartesian Product $$A \\times B = \\{(a, b) \\mid a \\in A,\\ b \\in B\\}, \\qquad |A \\times B| = |A| \\cdot |B|$$ Product Rule (PR) Also known as the rule of product. If a task is carried out in independent stages, with \\(|A_1|\\) ways for the 1st stage, \\(|A_2|\\) for the 2nd, \u0026hellip;, \\(|A_k|\\) for the \\(k\\)th, then the total number of ways to carry out the task is: $$|A_1 \\times A_2 \\times \\cdots \\times A_k| = |A_1| \\cdot |A_2| \\cdots |A_k|$$Application: when the choices are sequential and independent, that is, each stage does not interfere with the others.\nWorked Examples # The examples below progressively increase in complexity. In each one, we identify the applicable principle before calculating, and this habit avoids mistakes and makes the reasoning verifiable.\nPasswords with and without repetition # Problem: a lock has 8 available symbols. How many 2-symbol passwords exist: (a) with repeated symbols allowed; (b) with the two symbols necessarily distinct?\nPrinciple: PR in both cases, since the password is formed in two sequential stages (choose the 1st symbol, then the 2nd), and in each stage the number of options is constant regardless of the symbol chosen before.\n(a) With repetition: each position has 8 options, with no restriction between them:\n$$8 \\times 8 = 64 \\text{ passwords}$$(b) Without repetition: the 1st position still has 8 options, but the 2nd has only 7 (any symbol except the one already chosen):\n$$8 \\times 7 = 56 \\text{ passwords}$$Check by complement: of the 64 passwords with repetition, we remove the 8 where both positions have the same symbol (AA, BB, …): \\(64 - 8 = 56\\). ✓\nPython: itertools.product and itertools.permutations The itertools module in the standard library implements the Cartesian product directly, making the PR observable in code. The example below uses the case of 2-symbol passwords over an 8-character alphabet:\nimport itertools symbols = \u0026#34;ABCDEFGH\u0026#34; # 8 symbols # PR with repetition: Cartesian product 8 × 8 = 64 pairs with_rep = list(itertools.product(symbols, repeat=2)) print(len(with_rep)) # 64 # PR without repetition: each position excludes the symbol already used → 8 × 7 = 56 without_rep = list(itertools.permutations(symbols, 2)) print(len(without_rep)) # 56 itertools.product(A, B, C, ...) generates \\(|A| \\times |B| \\times |C| \\times \\cdots\\) tuples, exactly the PR. itertools.permutations(S, k) generates arrangements (r-permutations) of \\(k\\) distinct elements of \\(S\\) (distinct by position in \\(S\\), not by value), corresponding to the PR applied to stages without repetition. The function name anticipates a formal concept: a permutation is an ordered selection of distinct elements. The next article covers this topic in detail.\nApplication: brute force and the password space An attacker who tries every 8-character password from an alphabet of 95 printable symbols faces \\(95^8 \\approx 6.6 \\times 10^{15}\\) combinations, more than 6 quadrillion attempts. Forbidding repetition would reduce this space to \\(95 \\times 94 \\times \\cdots \\times 88\\), but human users choose passwords that are far from random, which reduces the effective space: that is why NIST SP 800-63B recommends comparing chosen passwords against lists of common, expected, or compromised values. The PR is the tool that formalizes this reasoning in password policies and attack cost estimates.\nFlag with 3 stripes of different colors # Problem: a flag has 3 horizontal stripes; there are 5 colors available and each stripe must have a color different from the others. How many distinct flags can be created?\nPrinciple: PR, since the assembly is done in 3 sequential stages (top, middle, and bottom stripe), and in each stage the number of available colors decreases predictably.\nStage Stripe Available options 1st Top 5 (any of the 5 colors) 2nd Middle 4 (any except the one used on the top stripe) 3rd Bottom 3 (any except the two already used) $$5 \\times 4 \\times 3 = 60 \\text{ flags}$$ Note The number of options in the 2nd stage is always 4, regardless of which color was chosen in the 1st: this constancy is exactly what validates the use of the PR.\nMultiple-choice exam # Problem: an exam has 20 questions. How many distinct answer keys are possible if each question is true or false? And if each question has 5 choices?\nPrinciple: PR, since each question is a stage independent of the others. The answer given to any question does not change the options of the others.\nTrue or false: each of the 20 questions has 2 options (T or F):\n$$\\underbrace{2 \\times 2 \\times \\cdots \\times 2}_{20\\text{ times}} = 2^{20} = 1{,}048{,}576 \\text{ answer keys}$$5 choices per question: the structure is identical, only each stage now has 5 options:\n$$5^{20} \\approx 9.5 \\times 10^{13} \\text{ answer keys}$$That is more than 95 trillion possible answer keys. Even trying one per second, it would take about 3 million years to exhaust them.\nApplication: generating test cases Imagine a system with \\(k\\) configuration parameters, each of which can take \\(n\\) values. Testing all possible combinations (the strategy called exhaustive coverage or full factorial) requires \\(n^k\\) test cases, exactly by the PR. For \\(k = 10\\) parameters with \\(n = 5\\) values each, that is already \\(5^{10} \\approx 10\\) million tests.\nIn practice, empirical research from NIST indicates that most defects are caused by the interaction of one or two parameters, with a smaller fraction requiring three or more. Pairwise testing (also known as all-pairs testing) takes advantage of this: instead of covering all \\(n^k\\) combinations, we choose a smaller subset of tests such that, for each pair of parameters, all \\(n^2\\) combinations of values for that pair appear at least once. So-called covering arrays achieve this with only \\(O(n^2 \\log k)\\) cases, a drastic reduction compared to full factorial.\n3-digit numbers with distinct digits from \\(\\{1,2,3,4,5,6\\}\\) # Problem: how many 3-digit numbers with distinct digits can be formed using only digits from \\(\\{1,2,3,4,5,6\\}\\)?\nPrinciple: PR, since we fill the 3 positions sequentially; in each position the number of options decreases by 1 because the chosen digit is not repeated.\nPosition Options Justification Hundreds 6 any of the 6 digits Tens 5 any except the hundreds digit Units 4 any except the two already used $$6 \\times 5 \\times 4 = 120 \\text{ numbers}$$Since 0 does not belong to the set, all the numbers formed are automatically valid 3-digit numbers: there is no risk of a leading zero.\n3-digit numbers with distinct digits from \\(\\{0,1,\\ldots,9\\}\\) # Problem: how many 3-digit numbers (between 100 and 999) have all distinct digits, using digits from \\(\\{0,1,\\ldots,9\\}\\)?\nPrinciple: PR, with the same structure as the previous example, but now 0 is available and cannot occupy the hundreds position (otherwise the number would have fewer than 3 digits).\nPosition Options Justification Hundreds 9 digits 1 to 9 (zero excluded) Tens 9 digits 0 to 9 except the hundreds digit (zero is an option again) Units 8 digits 0 to 9 except the two already used $$9 \\times 9 \\times 8 = 648 \\text{ numbers}$$Check by complement: we count all arrangements of 3 distinct digits (including those that start with zero) and subtract the invalid ones:\n$$\\underbrace{10 \\times 9 \\times 8}_{\\text{all arrangements with distinct digits}} - \\underbrace{1 \\times 9 \\times 8}_{\\text{start with 0}} = 720 - 72 = 648 \\checkmark$$The complement method always follows the same structure, useful when counting the invalid cases is simpler than counting the valid ones directly:\n\u0026#x2197; flowchart LR A[\"Total\nno restriction\"] --\u003e|\"− Invalid cases\n(easier to count)\"| B[\"Result\nonly the valid cases\"] Selection in a room # Problem: in a room there are 5 men, 6 women, and 4 children. In how many ways can we choose: (a) a single person; (b) one person of each type?\nThis example is instructive because the two sub-problems, with similar statements, require different principles.\n(a) A single person, by the Sum Rule:\nChoosing \u0026ldquo;either a man, or a woman, or a child\u0026rdquo;: the three categories are mutually exclusive (no one belongs to two groups simultaneously). By the SR:\n$$5 + 6 + 4 = 15 \\text{ ways}$$(b) One of each type, by the Product Rule:\nNow we want \u0026ldquo;a man and a woman and a child\u0026rdquo;: three independent choices in distinct groups. Whom we choose from the group of men does not interfere with whom we can choose among the women. By the PR:\n$$5 \\times 6 \\times 4 = 120 \\text{ ways}$$ SQL: UNION ALL (SR) vs CROSS JOIN (PR) The distinction \u0026ldquo;or is a sum, and is a product\u0026rdquo; has a direct counterpart in SQL. We use minimal tables to make the result visible:\nCREATE TABLE men (name TEXT); CREATE TABLE women (name TEXT); CREATE TABLE children (name TEXT); INSERT INTO men VALUES (\u0026#39;Bruno\u0026#39;), (\u0026#39;Carlos\u0026#39;); INSERT INTO women VALUES (\u0026#39;Ana\u0026#39;), (\u0026#39;Diana\u0026#39;); INSERT INTO children VALUES (\u0026#39;Edu\u0026#39;), (\u0026#39;Fabi\u0026#39;); SR, a single person (UNION ALL):\nSELECT name FROM men UNION ALL SELECT name FROM women UNION ALL SELECT name FROM children; name ------ Bruno Carlos Ana Diana Edu Fabi (6 rows: 2 + 2 + 2) PR, one of each type (CROSS JOIN):\nSELECT m.name AS man, w.name AS woman, c.name AS child FROM men m CROSS JOIN women w CROSS JOIN children c; man | woman | child -------|--------|-------- Bruno | Ana | Edu Bruno | Ana | Fabi Bruno | Diana | Edu Bruno | Diana | Fabi Carlos | Ana | Edu Carlos | Ana | Fabi Carlos | Diana | Edu Carlos | Diana | Fabi (8 rows: 2 × 2 × 2) UNION ALL stacks the rows of disjoint tables, adding the cardinalities. CROSS JOIN generates all possible combinations (pairs, triples, and so on), multiplying the cardinalities. This correspondence shows up whenever a relational schema needs to enumerate combinations among independent entities.\nDivisors of \\(N = 2^3 \\times 3^2 \\times 5^4\\) # Problem: how many positive divisors does \\(N = 2^3 \\times 3^2 \\times 5^4\\) have?\nKey observation: by the Fundamental Theorem of Arithmetic, every divisor of \\(N\\) necessarily has the form \\(2^\\alpha \\cdot 3^\\beta \\cdot 5^\\gamma\\), where the exponents satisfy:\n$$\\alpha \\in \\{0,1,2,3\\}, \\quad \\beta \\in \\{0,1,2\\}, \\quad \\gamma \\in \\{0,1,2,3,4\\}$$Principle: PR, since choosing \\(\\alpha\\), \\(\\beta\\), and \\(\\gamma\\) are three stages independent of each other. Any combination within the ranges above produces a valid divisor, distinct from all the others.\nStage Choice Options 1st \\(\\alpha\\) (exponent of 2) 4 values: 0, 1, 2, 3 2nd \\(\\beta\\) (exponent of 3) 3 values: 0, 1, 2 3rd \\(\\gamma\\) (exponent of 5) 5 values: 0, 1, 2, 3, 4 $$4 \\times 3 \\times 5 = 60 \\text{ divisors}$$Generalization: if \\(N = p_1^{a_1} \\cdots p_k^{a_k}\\) is the prime factorization of \\(N\\), the number of positive divisors is:\n$$(a_1+1)(a_2+1)\\cdots(a_k+1)$$The factor \\((a_i + 1)\\) counts the possible values for the exponent of \\(p_i\\): from \\(0\\) to \\(a_i\\), there are \\(a_i + 1\\) integers. Zero is included because a divisor may not contain that prime. For example, 9 is a divisor of \\(N\\) and corresponds to \\(\\alpha = 0, \\beta = 2, \\gamma = 0\\).\nPython: checking the formula against brute force For small \\(N\\) it is possible to confirm the result by enumerating all the divisors directly and comparing with the formula:\nfrom math import prod N = 2**3 * 3**2 * 5**4 # 45,000 # Brute force: list every d that divides N divisors = [d for d in range(1, N + 1) if N % d == 0] print(len(divisors)) # 60 # PR formula: product of (exponent + 1) exponents = [3, 2, 4] # exponents of 2, 3, and 5 print(prod(e + 1 for e in exponents)) # 60 For large \\(N\\) (as in cryptography, where the primes have hundreds of digits) brute force is infeasible, but the formula remains fast: it takes only \\(k\\) multiplications, one per prime factor, that is, \\(O(k)\\), provided the factorization is known. Finding that factorization for large \\(N\\) is, in itself, a computationally hard problem, and it is precisely on this hardness that the security of RSA (Rivest-Shamir-Adleman) rests, a classic asymmetric encryption algorithm, used in digital signatures and in HTTPS certificate authentication (key exchange via RSA existed in older versions of TLS, but was removed in TLS 1.3).\nSummary Table # Principle Typical situation Keyword Operation Sum Mutually exclusive choices \u0026ldquo;or\u0026rdquo; Sum Product Sequential stages \u0026ldquo;and\u0026rdquo; Product Exercises # Exercise 1: Round trip For a Rio-Belo Horizonte-Rio trip I can use a train, a bus, or a plane. In how many ways can I choose the transportation if I do not want to use the same mode on the return as on the way out?\nAnswer Outbound: 3 options. Return: 2 options (excluding the outbound one). By the PR: \\(3 \\times 2 = \\mathbf{6}\\) ways.\nExercise 2: 4-letter words How many 4-letter words with different letters can be formed from an alphabet of 26 letters?\nAnswer By the PR, 4 sequential stages with no repeated letter: 1st letter: 26 options; 2nd: 25 (any except the 1st); 3rd: 24; 4th: 23. \\(26 \\times 25 \\times 24 \\times 23 = \\mathbf{358{,}800}\\) words.\nExercise 3: Integers with distinct digits How many integers between 100 and 999 have distinct digits?\nAnswer 1st digit: 9 options (1 to 9); 2nd: 9 options (0 to 9, except the 1st); 3rd: 8 options. \\(9 \\times 9 \\times 8 = \\mathbf{648}\\)\nExercise 4: At least two equal digits How many 4-digit natural numbers have at least two equal digits?\nAnswer By complement:\nTotal number of 4-digit numbers: \\(9 \\times 10^3 = 9{,}000\\) With all digits distinct: \\(9 \\times 9 \\times 8 \\times 7 = 4{,}536\\) With at least two equal: \\(9{,}000 - 4{,}536 = \\mathbf{4{,}464}\\) Exercise 5: Numbers with digit restrictions How many 3-digit numbers with distinct digits, greater than 390, can be formed with \\(\\{0,1,2,3,4,5,6,7,8,9\\}\\)?\nAnswer Splitting into cases:\nStarts with 4 to 9 (6 digits): \\(6 \\times 9 \\times 8 = 432\\) Starts with 3: to exceed 390, the 2nd position must be 9 (1 option); 3rd position: 7 options, the digits \\(\\{1,2,4,5,6,7,8\\}\\) (excluding 0, since it would form 390, which does not satisfy the restriction, and 3 and 9, already used). Hence, \\(1 \\times 1 \\times 7 = 7\\) Total: \\(432 + 7 = \\mathbf{439}\\)\nExercise 6: Divisors of \\(M = a^m \\cdot b^n \\cdot c^p\\) How many divisors does \\(M = a^m \\cdot b^n \\cdot c^p\\) have (with \\(a, b, c\\) distinct primes)?\nAnswer Each divisor has the form \\(a^\\alpha b^\\beta c^\\gamma\\) with \\(0 \\leq \\alpha \\leq m\\), \\(0 \\leq \\beta \\leq n\\), \\(0 \\leq \\gamma \\leq p\\): $$(m+1)(n+1)(p+1) \\text{ divisors}$$ Next Steps # The sum and product rules are the foundation, but they have a limitation: they require the number of options at each stage to be independent of the earlier choices (or, when there is dependence, that the count of remaining options stay constant). When this condition is no longer satisfied, the analysis must be refined.\nThe natural next step is to count sequences without repetition in which all the elements of a set are used, the domain of permutations. After that, we will see how to count groupings where order does not matter (combinations) and how to handle additional restrictions that mix SR and PR in more elaborate ways.\n","date":"2026-06-30","externalUrl":null,"permalink":"/2026/06/sum-and-product-rules/","section":"Blog","summary":"With 4 math books and 3 Portuguese books, there are 7 ways to pick one book and 12 ways to pick one of each. The first count is a sum and the second is a product: these are the sum rule and the product rule, applied here to passwords, flags, answer keys, and counting divisors.","title":"Sum Rule and Product Rule: The Foundation of Counting","type":"posts"},{"content":" How many 8-character passwords are there? In how many ways can 10 people sit around a table? How many subsets does a set with 20 elements have?\nThese questions look simple until trying to list the cases one by one reveals the astronomical scale of the possibilities. This is where combinatorics comes in: the mathematics of counting efficiently.\nWhy does counting matter? # In computing and mathematics, counting discrete objects shows up everywhere:\nSecurity: the strength of a password depends on the number of possible combinations Algorithms: complexity analysis often involves counting paths or subsets Probability: computing probabilities requires knowing the number of favorable cases and the total Optimization: many problems involve picking the best among many possible configurations Combinatorics provides the tools to answer these questions exactly and efficiently, without listing everything.\nThe roadmap of this series # This series covers the foundations of combinatorics in increasing order of complexity. Each article brings precise definitions, worked examples and exercises with commented solutions.\nThe Sum Rule and the Product Rule # The starting point. The sum rule counts mutually exclusive situations (\u0026ldquo;or\u0026rdquo;) by adding. The product rule counts independent steps (\u0026ldquo;and\u0026rdquo;) by multiplying. All of combinatorics rests on these two rules.\n→ The Sum Rule and the Product Rule\nSimple and Circular Permutations # When we want to order all the elements of a set. In a row: \\(P_n = n!\\). In a circle: \\(PC_n = (n-1)!\\). Includes examples with restrictions (blocks, alternations, separations).\n→ Simple and Circular Permutations\nSimple Arrangements # Ordered selections of only \\(r \u0026lt; n\\) elements, also called r-permutations. The formula \\(A(n,r) = \\dfrac{n!}{(n-r)!}\\) shows up in podiums, routes and license plates.\n→ Simple Arrangements\nSimple Combinations # Unordered selections of \\(r\\) elements out of \\(n\\). The formula \\(C(n,r) = \\binom{n}{r}\\) solves problems about committees, card hands and samples. Order does not matter, only who was chosen.\n→ Simple Combinations\nPermutations with Repetition # What if the elements are not all distinct? The formula \\(\\dfrac{n!}{n_1! n_2! \\cdots n_r!}\\) adjusts the count by dividing out the overcounting. Applied to anagrams, binary sequences and lattice paths.\n→ Permutations with Repetition\nArrangements with Repetition # Ordered selections with repetition allowed: \\(AR(n,r) = n^r\\). Here \\(r\\) can be larger than \\(n\\). Shows up in passwords, number representation in computers and any sequence where symbols repeat.\n→ Arrangements with Repetition\nCombinations with Repetition # Unordered selections with repetition: \\(CR(n,r) = \\binom{n+r-1}{r}\\). The stars and bars method reveals the equivalence with non-negative integer solutions of equations.\n→ Combinations with Repetition\nBinomial Coefficients and Pascal\u0026rsquo;s Triangle # The identities of the coefficients \\(\\binom{n}{r}\\): symmetry, Pascal\u0026rsquo;s rule, the row theorem (\\(\\sum \\binom{n}{k} = 2^n\\)) and the column theorem. Pascal\u0026rsquo;s Triangle as the visual structure of these relations.\n→ Binomial Coefficients and Pascal\u0026#39;s Triangle\nThe Binomial Theorem # The theorem that joins algebra and combinatorics: \\((a+b)^n = \\sum_{r=0}^{n}\\binom{n}{r}a^{n-r}b^r\\). Includes the general term formula, examples of extracting specific coefficients and proofs of identities by substitution.\n→ The Binomial Theorem\nWhat comes before and after # This series is part of a larger sequence on Discrete Mathematics:\nBefore: The Language of Sets and From Base Case to Infinity, on set theory and mathematical induction After: The Mathematics of Connections, on graph theory Each series can be read on its own, but the progression was planned to build intuition gradually.\nQuick Reference Table # The topics covered throughout the series, with their main formulas, for quick reference:\nTechnique Order matters? Repetition? Formula Sum rule n/a n/a \\(|A| + |B|\\) (disjoint) Product rule n/a n/a \\(|A| \\cdot |B|\\) Simple permutation Yes No \\(n!\\) Circular permutation Yes No \\((n-1)!\\) Permutation w/ repetition Yes Yes (identical) \\(\\dfrac{n!}{n_1!\\cdots n_k!}\\) Simple arrangement Yes No \\(\\dfrac{n!}{(n-r)!}\\) Arrangement w/ repetition Yes Yes \\(n^r\\) Simple combination No No \\(\\dbinom{n}{r}\\) Combination w/ repetition No Yes \\(\\dbinom{n+r-1}{r}\\) When the problem is choosing \\(r\\) elements out of \\(n\\), two questions decide the formula: does order matter? Is repetition allowed? The figure below arranges the four answers in a grid and shows each case choosing 2 letters from \\(\\{a, b, c\\}\\):\nTwo questions, four formulas Enjoy the reading!\n","date":"2026-06-29","externalUrl":null,"permalink":"/2026/06/the-art-of-counting/","section":"Blog","summary":"How many 8-character passwords are there? In how many ways can 10 people sit around a table? Combinatorics answers without listing every case. This is the roadmap of the series: from the counting rules to permutations, arrangements, combinations and the binomial theorem.","title":"The Art of Counting: A Journey Through Combinatorics","type":"posts"},{"content":"Como provar afirmações sobre infinitos casos de uma só vez? Esta série cobre o Princípio de Indução Matemática, a Indução Forte e as Relações de Recorrência — ferramentas essenciais para raciocinar sobre sequências e algoritmos recursivos.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-06-25","externalUrl":null,"permalink":"/series/do-caso-base-ao-infinito/","section":"Séries","summary":"","title":"Do Caso Base ao Infinito","type":"series"},{"content":"","date":"2026-06-25","externalUrl":null,"permalink":"/tags/fibonacci/","section":"tags","summary":"","title":"Fibonacci","type":"tags"},{"content":"How do you prove statements about infinitely many cases all at once? This series covers the Principle of Mathematical Induction, Strong Induction and Recurrence Relations, essential tools for reasoning about sequences and recursive algorithms.\nThe articles are listed below, in the order they were written.\n","date":"2026-06-25","externalUrl":null,"permalink":"/series/from-base-case-to-infinity/","section":"Series","summary":"","title":"From Base Case to Infinity","type":"series"},{"content":"","date":"2026-06-25","externalUrl":null,"permalink":"/tags/recorrencia/","section":"tags","summary":"","title":"recorrência","type":"tags"},{"content":"","date":"2026-06-25","externalUrl":null,"permalink":"/tags/recurrence/","section":"tags","summary":"","title":"recurrence","type":"tags"},{"content":" You invest \\(1000\\) reais at \\(1\\%\\) per month: next month you have \\(1010\\), then \\(1020.10\\), then \\(1030.30\\ldots\\) On the other side, anyone who finances a car knows that this month\u0026rsquo;s outstanding balance is last month\u0026rsquo;s balance minus the payment, plus interest on what is still owed. In both cases, computing the value for month 12 requires knowing the one for month 11: there is no direct shortcut. Sequences like these, where each term depends on the previous ones, are called recurrence relations.\nBut the concept goes beyond finance. How many moves are needed to solve the Tower of Hanoi with \\(n\\) disks? A simple equation, \\(T_n = 2T_{n-1} + 1\\), contains the complete answer. That is the power of recurrence relations: describing an entire sequence from a local rule.\nIn this article we define the concept formally, see classic examples and learn the substitution method to obtain closed-form formulas, direct expressions that compute the \\(n\\)-th term without depending on the previous ones. At the end, we also see the characteristic equation, a technique that extends the repertoire to order-2 recurrences, and divide-and-conquer recurrences, with the Master Theorem.\nWhy Recurrence Relations? # Sequences defined in terms of themselves appear throughout computer science and discrete mathematics:\nAnalysis of recursive algorithms: the complexity of merge sort, quicksort and the Tower of Hanoi is naturally expressed by recurrences; solving them gives the algorithm\u0026rsquo;s \\(O\\) notation (see Divide-and-Conquer Recurrences). Combinatorial counting: the number of paths on a grid, triangulations of polygons and integer partitions satisfy recurrences that let us compute large terms from small ones. Growth modeling: rabbit populations (Fibonacci), growth with compound interest and basic epidemiological models are all recurrence relations. Dynamic programming: a technique for recurrences whose subproblems repeat. Each result is stored so it is not recomputed, and the cost becomes proportional to the number of distinct subproblems. In optimization problems, it also depends on the optimal solution being built from optimal solutions to the subproblems (optimal substructure). Formal languages and compilers: context-free grammars generate languages whose number of derivations satisfies recurrences; analyzing their complexity uses the same tools as this article. What Is a Recurrence Relation? # Definition A recurrence relation is a formula that relates a term \\(a_n\\) to some of its predecessors \\(a_{n-1}, a_{n-2}, \\ldots\\). Also called a difference equation. More at Recurrence relation.\nTo determine the sequence completely, we need the initial conditions, the values of \\(a_1, a_2, \\ldots\\) that serve as the starting point.\nA simple example: the partial sums \\(S_n = 1 + 2 + \\cdots + n\\) satisfy\n$$S_n = S_{n-1} + n, \\quad S_1 = 1$$At each step, we just add \\(n\\) to the previous value.\nExample: Valid Arithmetic Expressions # Consider expressions formed by digits (\\(0{-}9\\)) and arithmetic symbols (\\(+,-,\\times,\\div\\)), obeying two rules: the expression must begin and end with a digit, and two symbols cannot appear consecutively. We call such a sequence \u0026ldquo;valid\u0026rdquo; in the syntactic sense: the rules check only the structure of the string, not the mathematical meaning. So 5÷0 is counted as valid (it obeys both rules), even though division by zero is undefined.\nFor example:\nExpression Valid? Reason 0187 ✓ digits only 1+99 ✓ begins and ends with a digit, no consecutive symbols 5×3– ✗ ends with a symbol 6÷+2 ✗ two consecutive symbols (\\(\\div\\) and \\(+\\)) Let \\(N_k\\) be the number of valid expressions of length \\(k\\).\nInitial conditions: \\(N_0 = 0\\), \\(N_1 = 10\\) (the digits \\(0{-}9\\))\nBuilding the recurrence:\nTo obtain an expression of length \\(k\\) from shorter ones, there are two strategies:\nAppend a digit (10 options) to the right of an expression of length \\(k-1\\): contribution \\(10 N_{k-1}\\) Append a symbol + digit (\\(4 \\times 10 = 40\\) options) to the right of an expression of length \\(k-2\\): contribution \\(40 N_{k-2}\\) $$N_k = 10 N_{k-1} + 40 N_{k-2}$$The following diagram summarizes the two strategies and the resulting formula:\nCase I: an expression of length k−1 followed by a digit (10 options). Case II: an expression of length k−2 followed by an arithmetic symbol (4 options) and a digit (10 options). Checking for \\(k = 3\\) We want to count expressions of length 3. The two strategies produce:\nStrategy 1: append a digit to an expression of length 2 (\\(N_2 = 100\\) expressions, such as 01, 19, 53\u0026hellip;):\n$$10 \\times N_2 = 10 \\times 100 = 1{,}000$$Generated examples: 017, 195, 530\u0026hellip;\nStrategy 2: append a symbol + digit to an expression of length 1 (\\(N_1 = 10\\) expressions, the digits \\(0{-}9\\)):\n$$40 \\times N_1 = 40 \\times 10 = 400$$Generated examples: 1+9, 5×3, 0÷7\u0026hellip;\nTotal: \\(N_3 = 1{,}000 + 400 = 1{,}400\\) ✓\nComputing the first terms:\n\\(k\\) \\(N_k\\) 0 0 1 10 2 100 3 1,400 4 18,000 Why is there no closed-form formula in this section? In \\(N_k = 10N_{k-1} + 40N_{k-2}\\), each term depends on the two previous terms. We call this an order-2 recurrence. The examples that follow (Tower of Hanoi and others) depend only on the immediately preceding term (order 1), and that is the case where the substitution method presented in this article works well. For order-2 recurrences there is a specific technique, the characteristic equation, presented at the end of this article.\nThe Fibonacci Sequence: The Rabbit Problem # The classic Fibonacci problem (1202): a colony of rabbits grows according to these rules:\nIt starts with 1 newborn pair Every pair that is at least 2 months old produces 1 new pair per month No rabbit dies Let \\(F_n\\) be the number of pairs at the end of month \\(n\\). The pairs in month \\(n\\) are:\nThose that existed in the previous month (\\(F_{n-1}\\) pairs) Those newly born this month, which correspond to the pairs that were at least 2 months old in the previous month (\\(F_{n-2}\\) pairs) $$F_n = F_{n-1} + F_{n-2}, \\quad F_1 = 1,\\ F_2 = 1$$ \\(n\\) 1 2 3 4 5 6 7 8 9 10 11 12 13 \\(F_n\\) 1 1 2 3 5 8 13 21 34 55 89 144 233 At the end of 13 months: 233 pairs.\nThe family tree below shows how each adult pair produces a new pair every month, making it visible why \\(F_n = F_{n-1} + F_{n-2}\\):\n\u0026#x2197; flowchart TD subgraph m1 [\"Month 1: F₁ = 1\"] A[\"pair A (newborn)\"] end subgraph m2 [\"Month 2: F₂ = 1\"] B[\"pair A (adult)\"] end subgraph m3 [\"Month 3: F₃ = 2\"] C[\"pair A\"] D[\"pair B (newborn)\"] end subgraph m4 [\"Month 4: F₄ = 3\"] E[\"pair A\"] F[\"pair B (adult)\"] G[\"pair C (newborn)\"] end subgraph m5 [\"Month 5: F₅ = 5\"] H[\"pair A\"] I[\"pair B\"] J[\"pair C (adult)\"] K[\"pair D (newborn)\"] L[\"pair E (newborn)\"] end A --\u003e B B --\u003e C B --\u003e|produces| D C --\u003e E D --\u003e F C --\u003e|produces| G E --\u003e H F --\u003e I G --\u003e J E --\u003e|produces| K F --\u003e|produces| L Tower of Hanoi # The Tower of Hanoi puzzle was proposed by Édouard Lucas in 1883. We are given \\(n\\) disks of different sizes stacked in decreasing order on one peg. The goal is to transfer all of them to another peg, using an auxiliary peg, obeying:\nMove only one disk at a time (always from the top) Never place a larger disk on top of a smaller one Let \\(T_n\\) be the minimum number of moves for \\(n\\) disks.\nThe figure below shows the initial state (all disks on Peg 1) and the desired final state (all on Peg 2) for \\(n = 3\\):\nGreen disk = largest, purple = medium, orange = smallest. The goal is to transfer the 3 disks from Peg 1 to Peg 2. A concrete example: n = 3\nBefore thinking about formulas, let\u0026rsquo;s see what happens step by step with 3 disks. Each arrow in the figure below represents a single move: always the top disk of one peg to the top of another, respecting the rule of never placing a larger disk on a smaller one. Read the top row from left to right (States 0 → 3), go down to State 4, and read the bottom row from right to left (States 4 → 7):\nG = green disk (largest), P = purple disk (medium), O = orange disk (smallest). Each arrow represents exactly one move of a single disk. Reading order: left to right on the top row (States 0 to 3), then right to left on the bottom row (States 4 to 7). Exactly 7 moves were needed to transfer the 3 disks. To try other numbers of disks, there are interactive simulators, such as the one at Math is Fun.\nFinding the recursive pattern\nLooking closely at the 7 moves, we notice a structure in three blocks:\nMoves 1 to 3 (States 0 → 3): the 2 smaller disks go from Peg 1 to Peg 3, using Peg 2 as the auxiliary, in exactly \\(T_2 = 3\\) moves. Move 4 (State 3 → 4): the largest disk goes alone from Peg 1 to Peg 2, in 1 move. Moves 5 to 7 (States 4 → 7): the 2 smaller disks go from Peg 3 to Peg 2, using Peg 1 as the auxiliary, again \\(T_2 = 3\\) moves. Total: \\(T_3 = 3 + 1 + 3 = 7\\). ✓\nThe same reasoning applies to any \\(n\\): to move \\(n\\) disks, we always need to (1) solve the problem of size \\(n-1\\), (2) move the largest disk, and (3) solve the problem of size \\(n-1\\) again. The diagram below summarizes this recursive algorithm. Each block labeled \\(T_{n-1}\\) represents a complete subproblem with many individual moves, not a single step:\nStep 1: move the n-1 smaller disks to the auxiliary Peg (T_{n-1} individual moves). Step 2: move the largest disk to the destination Peg (1 move). Step 3: move the n-1 disks from the auxiliary to the destination (another T_{n-1} moves). Total: T_n = 2·T_{n-1} + 1. Why can\u0026rsquo;t it be done with fewer? The construction above shows that \\(2T_{n-1} + 1\\) moves are sufficient. They are also necessary, because the largest disk must leave the source peg at least once:\nAt the first move of the largest disk, the other \\(n-1\\) disks are all on the third peg: they cannot be on the largest disk\u0026rsquo;s peg, nor on the peg it is going to. Getting them there took at least \\(T_{n-1}\\) moves. At the last move of the largest disk, which leaves it at the destination, the other \\(n-1\\) are also all on a single peg, which is not the destination. Stacking them on top of the largest requires at least another \\(T_{n-1}\\) moves. Adding the largest disk\u0026rsquo;s own move, no solution takes fewer than \\(2T_{n-1} + 1\\) moves. With both bounds together, we formalize:\n$$T_n = 2T_{n-1} + 1, \\quad T_1 = 1$$Computing the first values:\n\\(n\\) 1 2 3 4 \\(T_n\\) 1 3 7 15 The call tree of \\(T(3)\\) makes the recurrence visible. Each node \\(T(k)\\) unfolds into two children \\(T(k-1)\\) plus one unit, until it reaches the base case \\(T(1) = 1\\):\n\u0026#x2197; flowchart TD T3[\"T(3) = 7\"] T2a[\"T(2) = 3\"] M1[\"+1\"] T2b[\"T(2) = 3\"] T1a[\"T(1) = 1\"] M2[\"+1\"] T1b[\"T(1) = 1\"] T1c[\"T(1) = 1\"] M3[\"+1\"] T1d[\"T(1) = 1\"] T3 --\u003e T2a \u0026 M1 \u0026 T2b T2a --\u003e T1a \u0026 M2 \u0026 T1b T2b --\u003e T1c \u0026 M3 \u0026 T1d The Substitution Method # The substitution method consists of applying the recurrence relation repeatedly, replacing each term by its own recurrence, until a general pattern emerges.\nEvery application of the method follows the same five steps:\nStep What to do 1. Write the recurrence Express \\(T(n)\\) in terms of \\(T(n-1)\\) (or \\(T(n-2)\\), etc.) with an initial condition 2. Substitute iteratively Replace \\(T(n-1)\\) by its own recurrence, then \\(T(n-2)\\), and so on 3. Recognize the pattern Generalize the expression for \\(T(n)\\) in terms of \\(T(n-k)\\) and the accumulated constants 4. Use the initial condition When \\(k = n-1\\) (or equivalent), substitute the base case to eliminate \\(T\\) 5. Verify by induction Prove that the formula obtained is correct for all \\(n\\) using mathematical induction The diagram below summarizes the flow of the method:\n\u0026#x2197; flowchart LR E1[\"1. Write\nthe recurrence\"] --\u003e E2[\"2. Substitute\niteratively\"] E2 --\u003e E3[\"3. Recognize\nthe pattern\"] E3 --\u003e E4[\"4. Use the\ninitial condition\"] E4 --\u003e E5[\"5. Verify\nby induction\"] E5 --\u003e FEC([\"closed-form formula ✓\"]) Solving the Tower of Hanoi # $$T_n = 2T_{n-1} + 1$$Substituting iteratively:\n$$T_n = 2T_{n-1} + 1 = 2(2T_{n-2}+1)+1 = 2^2 T_{n-2} + 2 + 1$$$$= 2^2(2T_{n-3}+1)+2+1 = 2^3 T_{n-3} + 2^2 + 2 + 1 = \\cdots$$$$= 2^{n-1} T_1 + 2^{n-2} + \\cdots + 2 + 1 = 2^{n-1} + \\sum_{k=0}^{n-2} 2^k = 2^{n-1} + (2^{n-1} - 1) = 2^n - 1$$Closed-form formula: \\(T_n = 2^n - 1\\)\nVerification by induction:\nBase case (\\(n=1\\)): \\(T_1 = 1 = 2^1 - 1\\) ✓\nIH: \\(T_k = 2^k - 1\\)\nInductive step (\\(n = k+1\\)):\n$$T_{k+1} = 2T_k + 1 \\stackrel{\\text{IH}}{=} 2(2^k - 1) + 1 = 2^{k+1} - 2 + 1 = 2^{k+1} - 1$$Therefore, \\(T_n = 2^n - 1\\), which is what we wanted to prove. \\(\\blacksquare\\)\nHistorical note Lucas proposed the problem with 64 disks, claiming that the end of the world would occur when the Brahmins of a temple in Benares finished moving them. With the formula: \\(T_{64} = 2^{64} - 1 \\approx 1.8 \\times 10^{19}\\) moves. Even at 1 move per second, it would take more than 580 billion years.\nLucas, beyond the Tower Édouard Lucas also gives his name to the Lucas sequence (2, 1, 3, 4, 7, 11, 18, …), which follows the same rule as Fibonacci, \\(L_n = L_{n-1} + L_{n-2}\\), with other initial conditions (\\(L_0 = 2\\), \\(L_1 = 1\\)). And to the Lucas-Lehmer test, created by him in 1878 and proved by Derrick Lehmer in 1930: a primality test for Mersenne numbers \\(2^p - 1\\) that is faster than general tests. That is why many of the largest known primes are Mersenne primes.\nApplication: Tower of Hanoi in Python The recursive solution directly implements the three steps of the algorithm:\ndef hanoi(n, source=\u0026#34;A\u0026#34;, destination=\u0026#34;C\u0026#34;, auxiliary=\u0026#34;B\u0026#34;): if n == 1: print(f\u0026#34;Move disk 1: {source} → {destination}\u0026#34;) return 1 total = hanoi(n - 1, source, auxiliary, destination) print(f\u0026#34;Move disk {n}: {source} → {destination}\u0026#34;) total += 1 + hanoi(n - 1, auxiliary, destination, source) return total print(hanoi(3)) # prints the 7 moves and returns 7 The proof by induction validates the code step by step:\nStep In the proof In the code Base case \\(T_1 = 1\\) if n == 1: return 1: 1 disk, 1 move IH \\(T_k = 2^k - 1\\) moves hanoi(n-1, ...) returns the correct value (by hypothesis) Inductive step \\(T_n = 2T_{n-1} + 1\\) total = hanoi(n-1,...) + 1 + hanoi(n-1,...) Calling hanoi(64) would take the same astronomical time computed above, and there is no shortcut: the minimality argument, made before the recurrence, shows that \\(2^n - 1\\) moves is the minimum possible for any algorithm, not just this one.\nExample: Ovals in the Plane # We are given \\(n\\) ovals in the plane obeying two conditions:\nAny two ovals cross at exactly 2 distinct points No point belongs to three or more ovals simultaneously (no triple point) Into how many regions do these \\(n\\) ovals divide the plane?\nThe figure below shows the cases \\(n = 1, 2, 3\\) with numbered regions and marked intersection points (colored dots = intersections between pairs of ovals):\nn=1: 2 regions (interior and exterior). n=2: 4 regions; red dots mark the 2 intersections between blue and magenta. n=3: 8 regions with 6 intersection points: red (blue∩magenta), green (blue∩orange) and violet (magenta∩orange). Gray = exterior region. Looking at the concrete cases:\n\\(k\\) Previous ovals Intersections with the previous ones New arcs New regions Total \\(a_k\\) 1 0 n/a n/a n/a 2 (base) 2 1 2 points 2 arcs 2 4 3 2 4 points 4 arcs 4 8 4 3 6 points 6 arcs 6 14 Why does each new oval add exactly \\(2(k-1)\\) regions?\nWhen the \\(k\\)-th oval enters the plane, it crosses each of the \\(k-1\\) previous ovals at exactly 2 points, creating \\(2(k-1)\\) intersection points on it. These points divide the boundary of the new oval into \\(2(k-1)\\) stretches of curve (called arcs): each one is the segment of the boundary between two consecutive intersection points.\nThe key is that each arc lies entirely within a single existing region (since the existing boundaries are crossed only at the intersection points). By crossing that region, the arc divides it in two, creating exactly 1 new region per arc.\nA concrete example (\\(k = 3\\), the orange oval in the \\(n=3\\) panel): The orange oval crosses the blue oval at 2 points (green dots) and the magenta oval at 2 points (violet dots), 4 points in total, which divide the orange boundary into 4 arcs. Each arc cuts one of the 4 existing regions in half: \\(4 + 4 = 8\\) regions. ✓\nLet \\(a_k\\) be the number of regions with \\(k\\) ovals:\n$$a_k = a_{k-1} + 2(k-1), \\quad a_1 = 2$$Solving by substitution:\n$$a_n = 2 + \\sum_{k=2}^{n} 2(k-1) = 2 + 2\\sum_{j=1}^{n-1} j = 2 + 2 \\cdot \\frac{(n-1)n}{2} = 2 + n(n-1)$$Closed-form formula: \\(a_n = n^2 - n + 2\\)\nChecking: \\(a_1 = 1 - 1 + 2 = 2\\) ✓, \\(a_2 = 4 - 2 + 2 = 4\\) ✓, \\(a_3 = 9 - 3 + 2 = 8\\) ✓, \\(a_4 = 16 - 4 + 2 = 14\\) ✓\nGoing Further: Order-2 Recurrences # The substitution method solves recurrences where each term depends only on the previous one. For linear homogeneous recurrences of order 2 (those in which each term is a linear combination of the two previous ones, with no independent terms) there is a systematic technique called the characteristic equation: we assume the solution has the form \\(a_n = r^n\\) and find which values of \\(r\\) satisfy the recurrence. The pipeline below summarizes the complete flow for the case of distinct roots (\\(\\Delta \\neq 0\\)):\n\u0026#x2197; flowchart LR R[\"linear homogeneous\nrecurrence of order 2\"] --\u003e S[\"assume\naₙ = rⁿ\"] S --\u003e Q[\"characteristic\nequation\"] Q --\u003e Ro[\"distinct roots\nr₁ ≠ r₂\"] Ro --\u003e G[\"general solution\nA·r₁ⁿ + B·r₂ⁿ\"] G --\u003e CI[\"initial\nconditions\"] CI --\u003e FC([\"closed-form formula ✓\"]) For repeated roots (\\(\\Delta = 0\\)), the correct form is \\((A + Bn) \\cdot r^n\\); this case does not occur in the examples of this article.\nFibonacci # The recurrence \\(F_n = F_{n-1} + F_{n-2}\\) with \\(F_1 = F_2 = 1\\).\nSubstituting \\(F_n = r^n\\) into the recurrence:\n$$r^n = r^{n-1} + r^{n-2}$$Dividing both sides by \\(r^{n-2}\\) (with \\(r \\neq 0\\)):\n$$r^2 = r + 1 \\implies r^2 - r - 1 = 0$$By the quadratic formula:\n$$r = \\frac{1 \\pm \\sqrt{1 + 4}}{2} = \\frac{1 \\pm \\sqrt{5}}{2}$$The two roots are \\(\\varphi = \\dfrac{1+\\sqrt{5}}{2} \\approx 1.618\\) (the golden ratio) and \\(\\psi = \\dfrac{1-\\sqrt{5}}{2} \\approx {-0.618}\\). Since both are independent solutions, the general solution is any linear combination of them:\n$$F_n = A \\varphi^n + B \\psi^n$$Using \\(F_1 = 1\\) and \\(F_2 = 1\\) to determine \\(A\\) and \\(B\\):\nStep 1: subtract the equations from the initial conditions:\n$$F_2 - F_1 = A(\\varphi^2 - \\varphi) + B(\\psi^2 - \\psi) = 0$$Since \\(\\varphi^2 - \\varphi = \\varphi(\\varphi - 1) = 1\\) and \\(\\psi^2 - \\psi = 1\\) (both roots satisfy \\(r^2 = r+1\\)), this gives \\(A + B = 0\\), hence \\(B = -A\\).\nStep 2: substitute \\(B = -A\\) into \\(F_1 = 1\\):\n$$A(\\varphi - \\psi) = 1 \\implies A = \\frac{1}{\\varphi - \\psi} = \\frac{1}{\\sqrt{5}}$$We obtain Binet\u0026rsquo;s formula:\n$$F_n = \\frac{\\varphi^n - \\psi^n}{\\sqrt{5}}$$The previous article in the series proves this formula by strong induction.\nArithmetic Expressions # The recurrence \\(N_k = 10N_{k-1} + 40N_{k-2}\\) with \\(N_0 = 0\\) and \\(N_1 = 10\\).\nSubstituting \\(N_k = r^k\\) and dividing by \\(r^{k-2}\\):\n$$r^2 - 10r - 40 = 0 \\implies r = \\frac{10 \\pm \\sqrt{100 + 160}}{2} = 5 \\pm \\sqrt{65}$$The general solution is \\(N_k = A(5+\\sqrt{65})^k + B(5-\\sqrt{65})^k\\). Using the initial conditions:\n\\(N_0 = 0 \\implies A + B = 0 \\implies B = -A\\) \\(N_1 = 10 \\implies A \\cdot 2\\sqrt{65} = 10 \\implies A = \\dfrac{\\sqrt{65}}{13}\\) Closed-form formula:\n$$N_k = \\frac{\\sqrt{65}}{13}\\left[\\left(5+\\sqrt{65}\\right)^k - \\left(5-\\sqrt{65}\\right)^k\\right]$$Checking against the table:\n\\(N_1 = \\dfrac{\\sqrt{65}}{13} \\cdot 2\\sqrt{65} = \\dfrac{130}{13} = 10\\) ✓ \\(N_2 = \\dfrac{\\sqrt{65}}{13} \\cdot 20\\sqrt{65} = \\dfrac{1300}{13} = 100\\) ✓ The contrast with Fibonacci is instructive: the same technique produces results of very different elegance. Closed-form formulas always exist for linear recurrences with constant coefficients, written from the roots of the characteristic equation, but they are not always pretty.\nDivide-and-Conquer Recurrences # All the recurrences so far step back one or two terms: \\(T_{n-1}\\), \\(T_{n-2}\\). Divide-and-conquer algorithms do something else: they break the input into parts that are a fraction of the original size. Merge sort, seen in the article on strong induction, splits the list in half, sorts each half and merges the two sorted halves in time proportional to \\(n\\). Counting the cost \\(T(n)\\) for \\(n\\) elements:\n$$T(n) = 2T(n/2) + n, \\quad T(1) = 1$$To keep the calculations exact, we take \\(n\\) to be a power of 2; for other values of \\(n\\), the roundings of \\(n/2\\) change the constants, but not the growth.\nSolving by substitution # With \\(n = 2^m\\), the recurrence steps back through the powers of 2 and the substitution method works as before:\n$$T(n) = 2T(n/2) + n = 2\\left[2T(n/4) + \\frac{n}{2}\\right] + n = 4T(n/4) + 2n$$$$= 4\\left[2T(n/8) + \\frac{n}{4}\\right] + 2n = 8T(n/8) + 3n = \\cdots = 2^i T(n/2^i) + i \\cdot n$$The substitution stops at \\(i = m = \\log_2 n\\), when \\(n/2^m = 1\\):\n$$T(n) = 2^m T(1) + m \\cdot n = n + n \\log_2 n$$Closed-form formula: \\(T(n) = n \\log_2 n + n\\); therefore, \\(T(n) = O(n \\log n)\\), as announced in the series introduction article.\nThe recursion tree shows where each term comes from. In the figure below, for \\(n = 8\\), each level doubles the number of subproblems and halves the size of each one, so the merges at each level always add up to 8; there are \\(\\log_2 8 = 3\\) levels of merging, and the 8 leaves add another 8:\nT(8) = 32: three levels of merging costing 8 each, plus 8 leaves with T(1) = 1; in general, T(n) = n log₂ n + n Verification by induction (on \\(m\\), with \\(n = 2^m\\)):\nBase case (\\(m = 0\\), \\(n = 1\\)): \\(T(1) = 1 = 1 \\cdot 0 + 1\\) ✓\nIH: \\(T(2^k) = 2^k \\cdot k + 2^k\\)\nInductive step (\\(m = k+1\\)):\n$$T(2^{k+1}) = 2T(2^k) + 2^{k+1} \\stackrel{\\text{IH}}{=} 2\\left(2^k k + 2^k\\right) + 2^{k+1} = 2^{k+1}(k+1) + 2^{k+1}$$Therefore, \\(T(n) = n \\log_2 n + n\\) for every power of 2, \\(n = 2^m\\), which is what we wanted to prove. \\(\\blacksquare\\)\nThe Master Theorem # The same reasoning holds for any algorithm that splits the input into \\(a\\) subproblems of size \\(n/b\\) and spends extra work of the order of \\(n^d\\) to divide and combine. The result is the Master Theorem, here in its most commonly used form:\nMaster Theorem (simplified form) If \\(T(n) = aT(n/b) + \\Theta(n^d)\\), with constants \\(a \\geq 1\\), \\(b \u0026gt; 1\\) and \\(d \\geq 0\\), then:\n\\(T(n) = \\Theta(n^d)\\), if \\(d \u0026gt; \\log_b a\\); \\(T(n) = \\Theta(n^d \\log n)\\), if \\(d = \\log_b a\\); \\(T(n) = \\Theta(n^{\\log_b a})\\), if \\(d \u0026lt; \\log_b a\\). The \\(\\Theta\\) notation indicates exact growth: the function stays between two constant multiples of \\(n^d\\) (or of whatever is inside the \\(\\Theta\\)), above and below.\nThe intuition comes from the recursion tree. At each level, the number of subproblems is multiplied by \\(a\\) and the work of each one by \\(1/b^d\\), that is, the total work of the level is multiplied by \\(a/b^d\\). If \\(a/b^d \u0026lt; 1\\) (that is, \\(d \u0026gt; \\log_b a\\)), the work shrinks at each level and the root dominates. If \\(a/b^d = 1\\), all \\(\\log_b n\\) levels cost the same. If \\(a/b^d \u0026gt; 1\\), the work grows at each level and the \\(n^{\\log_b a}\\) leaves dominate.\nAlgorithm Recurrence \\(a\\), \\(b\\), \\(d\\) Case Result Merge sort \\(T(n) = 2T(n/2) + \\Theta(n)\\) 2, 2, 1 \\(d = \\log_2 2 = 1\\) \\(\\Theta(n \\log n)\\) Binary search \\(T(n) = T(n/2) + \\Theta(1)\\) 1, 2, 0 \\(d = \\log_2 1 = 0\\) \\(\\Theta(\\log n)\\) Karatsuba multiplication \\(T(n) = 3T(n/2) + \\Theta(n)\\) 3, 2, 1 \\(d \u0026lt; \\log_2 3 \\approx 1.58\\) \\(\\Theta(n^{\\log_2 3})\\) Quadratic merge (hypothetical) \\(T(n) = 2T(n/2) + \\Theta(n^2)\\) 2, 2, 2 \\(d \u0026gt; \\log_2 2 = 1\\) \\(\\Theta(n^2)\\) Binary search discards half of the interval at each step and spends constant time deciding which half stays. Karatsuba\u0026rsquo;s algorithm multiplies two numbers of \\(n\\) digits with three multiplications of numbers of \\(n/2\\) digits, instead of the four of the grade-school method. The last row shows what would happen with a merge sort whose merge was quadratic: the work at the root would come to dominate.\nWhen the theorem does not apply The simplified form requires subproblems of the same size \\(n/b\\) and extra work of the form \\(n^d\\). Recurrences such as \\(T(n) = T(n-1) + n\\) (which steps back one term, not a fraction) or \\(T(n) = 2T(n/2) + n \\log n\\) are left out: the first is solved by substitution, as at the beginning of this article, and the second needs the general form of the theorem, described in the entry cited above.\nChecking the formulas in Python Every closed-form formula in this article can be checked by iterating the recurrence and comparing the terms:\nfrom math import isclose, sqrt # Tower of Hanoi: T1 = 1, Tn = 2 T(n-1) + 1 T = {1: 1} for n in range(2, 31): T[n] = 2 * T[n - 1] + 1 print(all(T[n] == 2**n - 1 for n in T)) # True # Ovals in the plane: a1 = 2, an = a(n-1) + 2(n-1) a = {1: 2} for n in range(2, 31): a[n] = a[n - 1] + 2 * (n - 1) print(all(a[n] == n**2 - n + 2 for n in a)) # True # Arithmetic expressions: N0 = 0, N1 = 10, Nk = 10 N(k-1) + 40 N(k-2) N = {0: 0, 1: 10} for k in range(2, 21): N[k] = 10 * N[k - 1] + 40 * N[k - 2] r1, r2 = 5 + sqrt(65), 5 - sqrt(65) print(all(isclose(N[k], sqrt(65) / 13 * (r1**k - r2**k)) for k in N)) # True # Merge sort, with n = 2^m: T(1) = 1, T(n) = 2 T(n/2) + n M = {0: 1} for m in range(1, 31): M[m] = 2 * M[m - 1] + 2**m print(all(M[m] == 2**m * m + 2**m for m in M)) # True The comparison with isclose for the arithmetic expressions is necessary because the closed-form formula involves \\(\\sqrt{65}\\), which the computer stores only approximately.\nSummary Table # The results proved in this article, gathered for quick reference:\nSequence Base Recurrence Closed-Form Formula Fibonacci \\(F_1 = F_2 = 1\\) \\(F_n = F_{n-1} + F_{n-2}\\) \\(F_n = \\dfrac{\\varphi^n - \\psi^n}{\\sqrt{5}}\\) (Binet) Tower of Hanoi \\(T_1 = 1\\) \\(T_n = 2T_{n-1} + 1\\) \\(T_n = 2^n - 1\\) Ovals in the plane \\(a_1 = 2\\) \\(a_n = a_{n-1} + 2(n-1)\\) \\(a_n = n^2 - n + 2\\) Partial sums \\(S_1 = 1\\) \\(S_n = S_{n-1} + n\\) \\(S_n = n(n+1)/2\\) Arithmetic expressions \\(N_0 = 0,\\ N_1 = 10\\) \\(N_k = 10N_{k-1} + 40N_{k-2}\\) \\(N_k = \\dfrac{\\sqrt{65}}{13}\\left[(5+\\sqrt{65})^k - (5-\\sqrt{65})^k\\right]\\) Merge sort (\\(n = 2^m\\)) \\(T(1) = 1\\) \\(T(n) = 2T(n/2) + n\\) \\(T(n) = n \\log_2 n + n\\) Exercises # Practice setting up and solving recurrences with the substitution method. Exercises 1 and 2 follow the model of the examples seen; exercises 3 and 4 demand more attention to setting up the recurrence and to the algebraic verification.\nExercise 1: Double Tower of Hanoi A variant of Hanoi: \\(2n\\) disks of \\(n\\) sizes, two of each size. The rules are the same as in the classic Hanoi.\n(a) Set up the recurrence relation. (b) Solve it by the substitution method.\nSolutions (a) To move \\(2n\\) disks: move the \\(2(n-1)\\) smallest (\\(T(n-1)\\) moves), move the 2 largest (2 moves), move the \\(2(n-1)\\) smallest back (\\(T(n-1)\\) moves). $$T(1) = 2, \\quad T(n) = 2T(n-1) + 2 \\text{ for } n \\geq 2$$(b) By iterative substitution: $$T(n) = 2T(n-1) + 2 = 2^i T(n-i) + \\sum_{k=1}^i 2^k$$For \\(i = n-1\\): $$T(n) = 2^{n-1} \\cdot T(1) + \\sum_{k=1}^{n-1} 2^k = 2^n + (2^n - 2) = \\mathbf{2^{n+1} - 2}$$ Exercise 2: Unbounded regions formed by lines Let \\(a_n\\) be the number of unbounded regions created by \\(n\\) lines in general position (no parallels and no triple point).\n(a) Illustrate for \\(n = 0, 1, 2, 3, 4\\). (b) Set up and solve the recurrence.\nSolutions (a) From the second line on, each new line adds 2 unbounded regions, one at each end. The first is an exception, because it divides the whole plane (1 region) into two half-planes, an increase of only 1 region:\n\\(n\\) \\(a_n\\) 0 1 1 2 2 4 3 6 4 8 The figure below illustrates the five cases, with the unbounded regions numbered around the frame and the bounded ones in gray, which do not count:\nUnbounded regions formed by n lines in general position: 1, 2, 4, 6 and 8 for n = 0 to 4; the bounded ones (0, 0, 0, 1 and 3) appear in gray (b) \\(a_n = a_{n-1} + 2\\) for \\(n \\geq 2\\), with \\(a_1 = 2\\). The case \\(a_0 = 1\\) does not follow the rule, because going from 0 to 1 line the increase is 1.\nBy substitution: \\(a_n = a_1 + 2(n-1) = 2 + 2n - 2 = \\mathbf{2n}\\)\nExercise 3: Verification by induction Given the recurrence \\(a_n = a_{n-1} + 2a_{n-2}\\) with \\(a_1 = 1\\) and \\(a_2 = 3\\), verify by strong induction that the closed-form formula is:\n$$a_n = \\frac{1}{3}(-1)^n + \\frac{2}{3} \\cdot 2^n$$ Proof Base case: \\(a_1 = \\dfrac{-1}{3} + \\dfrac{4}{3} = 1\\) ✓ and \\(a_2 = \\dfrac{1}{3} + \\dfrac{8}{3} = 3\\) ✓\nSIH: the formula holds for all \\(i \\leq k\\) with \\(k \\geq 2\\).\nInductive step (\\(n = k+1\\)):\n$$a_{k+1} = a_k + 2a_{k-1} \\stackrel{\\text{SIH}}{=} \\left[\\frac{1}{3}(-1)^k + \\frac{2}{3} \\cdot 2^k\\right] + 2\\left[\\frac{1}{3}(-1)^{k-1} + \\frac{2}{3} \\cdot 2^{k-1}\\right]$$$$= \\frac{1}{3}\\bigl[(-1)^k + 2(-1)^{k-1}\\bigr] + \\frac{2}{3}\\bigl[2^k + 2^k\\bigr]$$$$= \\frac{1}{3}(-1)^{k-1}(-1+2) + \\frac{2}{3} \\cdot 2^{k+1} = \\frac{1}{3}(-1)^{k+1} + \\frac{2}{3} \\cdot 2^{k+1}$$Therefore, \\(a_n = \\dfrac{1}{3}(-1)^n + \\dfrac{2}{3} \\cdot 2^n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 4: Immortal plant A plant lives forever, but reproduces only once, right after its 1st year of life. Starting with 1 plant in year 1, how many plants are there in year \\(n\\)?\n(a) Write the recurrence relation. (b) Solve it and give the closed-form formula.\nSolutions (a) In year \\(n\\), the plant from year \\(n-1\\) produces 1 new plant, while all the others stay alive: $$a_1 = 1, \\quad a_n = a_{n-1} + 1 \\text{ for } n \\geq 2$$(b) By substitution: \\(a_n = a_1 + (n-1) = \\mathbf{n}\\)\nNext Steps # With induction (simple and strong) and recurrence relations in your repertoire, you have the fundamental tools to reason about recursively defined structures. The next step is combinatorics, the mathematics of counting. The next series, The Art of Counting, uses these foundations to build the theory of permutations, arrangements, combinations and the binomial theorem.\n","date":"2026-06-25","externalUrl":null,"permalink":"/2026/06/recurrence-relations/","section":"Blog","summary":"How many moves does it take to solve the Tower of Hanoi? The answer comes from a recurrence relation, which writes the problem in terms of itself with one disk fewer. We set up recurrences and solve them by substitution, by the characteristic equation and by the Master Theorem, used for merge sort.","title":"Recurrence Relations: Defining Sequences in Terms of Themselves","type":"posts"},{"content":"","date":"2026-06-23","externalUrl":null,"permalink":"/tags/inducao-matematica/","section":"tags","summary":"","title":"indução matemática","type":"tags"},{"content":"","date":"2026-06-23","externalUrl":null,"permalink":"/tags/mathematical-induction/","section":"tags","summary":"","title":"mathematical induction","type":"tags"},{"content":"","date":"2026-06-23","externalUrl":null,"permalink":"/tags/mathematical-proof/","section":"tags","summary":"","title":"mathematical proof","type":"tags"},{"content":"","date":"2026-06-23","externalUrl":null,"permalink":"/tags/prova-matematica/","section":"tags","summary":"","title":"prova matemática","type":"tags"},{"content":" In the previous article, the Principle of Mathematical Induction (PMI) assumed only \\(P(k)\\) to prove \\(P(k+1)\\). That is enough in many cases. But what if step \\(k+1\\) depends not only on \\(k\\), but on two or more earlier cases at the same time?\nThe answer is Strong Induction, a variant that assumes the truth of all earlier cases at once.\nWhy Strong Induction? # Simple induction is enough when each step depends only on the immediately preceding case. Strong induction is needed when the reasoning has to reach back more than one level, which happens often in computer science:\nMulti-step recurrent sequences: Fibonacci uses \\(F_{k+1} = F_k + F_{k-1}\\); proving any property of this kind of sequence by induction requires both earlier cases. Divide-and-conquer algorithms: merge sort and quick sort split the problem into parts of arbitrary size (not necessarily \\(k-1\\)); the correctness proof needs the hypothesis for all smaller subproblems. Factorization and number theory: the proof that every integer \\(n \u0026gt; 1\\) has a prime factor requires the hypothesis for any divisor of \\(n\\), which can be much smaller than \\(n-1\\). Structural properties of trees: proofs about a tree with \\(n\\) nodes often fall back on subtrees of any size smaller than \\(n\\). This article presents the technique with examples of increasing complexity, culminating in Binet\u0026rsquo;s Formula for Fibonacci.\nThe Fibonacci Sequence # Before the formal principle, it is worth introducing the sequence that will be used as an example throughout this article.\nFibonacci Sequence $$F_1 = 1,\\quad F_2 = 1,\\quad F_n = F_{n-1} + F_{n-2}\\ \\text{for}\\ n \\geq 3$$First terms: \\(1, 1, 2, 3, 5, 8, 13, 21, 34, 55, \\ldots\\)\nEach term is the sum of the two before it:\n$$F_3 = F_2 + F_1 = 2, \\quad F_4 = F_3 + F_2 = 3, \\quad F_5 = F_4 + F_3 = 5, \\ldots$$ An observed identity # Computing the first cases:\n\\(n\\) \\(F_1^2 + \\cdots + F_n^2\\) \\(F_n \\cdot F_{n+1}\\) 1 1 \\(1 \\cdot 1 = 1\\) 2 2 \\(1 \\cdot 2 = 2\\) 3 6 \\(2 \\cdot 3 = 6\\) 4 15 \\(3 \\cdot 5 = 15\\) The pattern suggests: \\(F_1^2 + F_2^2 + \\cdots + F_n^2 = F_n \\cdot F_{n+1}\\).\nThis identity can be proved by simple induction (each step only needs the previous case):\nBase (\\(n = 1\\)): \\(F_1^2 = 1 = F_1 \\cdot F_2\\) ✓\nIH: \\(F_1^2 + \\cdots + F_k^2 = F_k \\cdot F_{k+1}\\)\nInductive step (\\(n = k+1\\)):\n$$F_1^2 + \\cdots + F_k^2 + F_{k+1}^2 \\stackrel{\\text{IH}}{=} F_k F_{k+1} + F_{k+1}^2 = F_{k+1}(F_k + F_{k+1}) = F_{k+1} \\cdot F_{k+2}$$Therefore, \\(F_1^2 + F_2^2 + \\cdots + F_n^2 = F_n \\cdot F_{n+1}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nApplication: recursive Fibonacci and exponential complexity The naive recursive implementation of Fibonacci in Python directly mirrors the definition:\ndef fib(n): if n \u0026lt;= 2: return 1 return fib(n - 1) + fib(n - 2) Each call generates two smaller calls (fib(n-1) and fib(n-2)), and many subproblems are recomputed over and over. Letting \\(C_n\\) be the number of calls made by fib(n), we have \\(C_1 = C_2 = 1\\) and \\(C_n = 1 + C_{n-1} + C_{n-2}\\), and a strong induction shows that \\(C_n = 2F_n - 1\\). The cost therefore grows like \\(F_n\\) itself, and \\(F_n\\) grows exponentially: by Binet\u0026rsquo;s Formula, proved in Exercise 4, \\(F_n \\approx \\varphi^n/\\sqrt{5}\\), with \\(\\varphi \\approx 1.618\\). For \\(n = 50\\), that is more than 25 billion calls. Example 1, below, gives the other side of the calculation: the upper bound \\(F_n \u0026lt; \\left(\\frac{7}{4}\\right)^n\\) guarantees that the growth is no worse than exponential.\nThe solution is memoization, which stores each computed \\(F_k\\) and reuses it:\nfrom functools import lru_cache @lru_cache(maxsize=None) def fib(n): if n \u0026lt;= 2: return 1 return fib(n - 1) + fib(n - 2) With memoization, each value is computed only once, with complexity \\(O(n)\\).\nThe impact shows up in practical measurements. For \\(n = 35\\), with Python 3.13 on a recent personal computer:\nVersion Time (1 run) Complexity Naive ~0.75 s exponential Memoized ~0.01 ms \\(O(n)\\) Binet ~0.01 ms \\(O(1)\\) The exact times vary from machine to machine; what repeats is the ratio. Each extra unit in \\(n\\) multiplies the time of the naive version by about \\(\\varphi \\approx 1.6\\): from \\(n = 30\\) to \\(n = 35\\), the time becomes about ten times larger.\nBinet\u0026rsquo;s Formula, which will be proved below in Exercise 4, reduces the computation to \\(O(1)\\) arithmetic operations, and strong induction is the mathematical guarantee that the formula is correct for every \\(n\\). But the guarantee is mathematical: the computer does not work with an exact \\(\\sqrt{5}\\). In 64-bit floating point, the formula is correct up to \\(F_{70}\\) and fails from \\(F_{71}\\) on:\nfrom math import sqrt phi = (1 + sqrt(5)) / 2 psi = (1 - sqrt(5)) / 2 def fib_binet(n): return round((phi**n - psi**n) / sqrt(5)) def fib_exact(n): a, b = 0, 1 for _ in range(n): a, b = b, a + b return a print(fib_binet(70) == fib_exact(70)) # True print(fib_binet(71)) # 308061521170130 print(fib_exact(71)) # 308061521170129 The float64 keeps about 16 significant digits, and \\(F_{71}\\) already has 15: the small rounding error in \\(\\sqrt{5}\\) and in \\(\\varphi^n\\) now reaches the units digit. The article on the IEEE 754 standard explains where this limit comes from. For exact values with large \\(n\\), the versions that use only integers (memoized or iterative) are the right choice.\nThe redundancy of the computation is evident in the call tree of fib(5): fib(3) is computed twice and fib(2) three times, and the problem worsens exponentially with \\(n\\).\n\u0026#x2197; flowchart TD F5[\"fib(5)\"] --\u003e F4[\"fib(4)\"] F5 --\u003e F3a[\"fib(3)\"] F4 --\u003e F3b[\"fib(3)\"] F4 --\u003e F2a[\"fib(2)\"] F3a --\u003e F2b[\"fib(2)\"] F3a --\u003e F1a[\"fib(1)\"] F3b --\u003e F2c[\"fib(2)\"] F3b --\u003e F1b[\"fib(1)\"] Principle of Strong Induction # Strong Induction Let \\(P(n)\\) be a property of \\(n \\in \\mathbb{N}\\). If:\nBase: \\(P(1)\\) is true Step: \\(\\bigl[P(1), P(2), \\ldots, P(k)\\ \\text{are all true}\\bigr] \\Rightarrow P(k+1)\\) Then \\(P(n)\\) is true for every \\(n \\in \\mathbb{N}\\).\nEquivalence with the PMI The Principle of Strong Induction is equivalent to the standard PMI: any proposition that can be proved by one can also be proved by the other. The difference is practical: the stronger hypothesis of strong induction makes certain proofs much more direct.\nThe difference lies in the hypothesis: in the PMI, we assume only \\(P(k)\\). In strong induction, also called complete induction, we assume \\(P(1), P(2), \\ldots, P(k)\\), all earlier cases simultaneously. This is particularly useful when \\(P(k+1)\\) depends on more distant cases (such as \\(P(k-1)\\), \\(P(k-2)\\), etc.).\nThe diagram below contrasts the two chains of implication: in simple induction, each node receives an arrow only from the previous one; in strong induction, each node receives arrows from all the previous ones.\n\u0026#x2197; flowchart LR subgraph simples[\"Simple Induction\"] direction LR P1[\"P(1)\"] --\u003e P2[\"P(2)\"] P2 --\u003e P3[\"P(3)\"] P3 --\u003e P4[\"P(4)\"] P4 --\u003e PN[\"...\"] end subgraph forte[\"Strong Induction\"] direction LR Q1[\"P(1)\"] --\u003e Q3[\"P(3)\"] Q2[\"P(2)\"] --\u003e Q3 Q1 --\u003e Q4[\"P(4)\"] Q2 --\u003e Q4 Q3 --\u003e Q4 Q4 --\u003e QN[\"...\"] end Structure of the proof # Every proof by strong induction follows the same three-step structure:\nStep What to do Base Verify \\(P(1)\\) (and sometimes \\(P(2)\\) or more) Strong hypothesis (SIH) Assume \\(P(1), P(2), \\ldots, P(k)\\) are true for an arbitrarily fixed \\(k \\geq 1\\) Inductive step Prove \\(P(k+1)\\) using the SIH How many base cases are needed? When the recurrence in the inductive step reaches back \\(r\\) positions (that is, \\(P(k+1)\\) depends on \\(P(k), P(k-1), \\ldots, P(k-r+1)\\)), you must verify \\(r\\) base cases independently. For Fibonacci (\\(r = 2\\)), the cases \\(P(1)\\) and \\(P(2)\\) are needed; for a recurrence of order 3, they would be \\(P(1)\\), \\(P(2)\\) and \\(P(3)\\). Without these initial cases, the inductive chain does not have a sufficient starting point.\nWhat if there is no base case? There is a leaner formulation of strong induction: if, for every \\(n \\geq 1\\), the truth of \\(P(m)\\) for all \\(m \u0026lt; n\\) implies \\(P(n)\\), then \\(P(n)\\) holds for every \\(n\\). It seems to dispense with the base, but it does not: for \\(n = 1\\) there is no \\(m \u0026lt; 1\\), the hypothesis is empty, and the implication requires proving \\(P(1)\\) with no help at all, which is precisely the base case. The version with an explicit base, used in this article, just keeps that step in plain view.\nApplication: Merge sort and correctness by divide and conquer Merge sort sorts a list by splitting it in half, sorting each half recursively and merging the results:\ndef merge(a, b): result, i, j = [], 0, 0 while i \u0026lt; len(a) and j \u0026lt; len(b): if a[i] \u0026lt;= b[j]: result.append(a[i]); i += 1 else: result.append(b[j]); j += 1 return result + a[i:] + b[j:] def merge_sort(lst): if len(lst) \u0026lt;= 1: return lst mid = len(lst) // 2 left = merge_sort(lst[:mid]) right = merge_sort(lst[mid:]) return merge(left, right) The correctness proof follows exactly the structure of strong induction:\nStep In merge sort Base len(lst) \u0026lt;= 1: an empty or one-element list is already sorted SIH merge_sort(lst[:mid]) and merge_sort(lst[mid:]) return sorted lists (subproblems of sizes \\(\\lfloor n/2 \\rfloor\\) and \\(\\lceil n/2 \\rceil\\), both \\(\\leq k\\)) Step merge(left, right): merging two sorted lists produces a sorted list Simple hypothesis would not be enough here: the subproblems have sizes \\(\\lfloor n/2 \\rfloor\\) and \\(\\lceil n/2 \\rceil\\), not necessarily \\(n-1\\). The SIH guarantees correctness for any subproblem size smaller than \\(n\\).\nAs for the cost, merge sort satisfies \\(T(n) = 2T(n/2) + n\\), solved, with the answer \\(O(n \\log n)\\), in the section on divide and conquer of the article on recurrence relations.\nThe call tree of merge_sort([5, 3, 8, 1]) contrasts with the Fibonacci one: each subproblem is computed only once and the results are combined from the bottom up by the merges.\n\u0026#x2197; flowchart TD A[\"[5, 3, 8, 1]\"] --\u003e|split| B[\"[5, 3]\"] A --\u003e|split| C[\"[8, 1]\"] B --\u003e|split| D[\"[5]\"] B --\u003e|split| E[\"[3]\"] C --\u003e|split| F[\"[8]\"] C --\u003e|split| G[\"[1]\"] D \u0026 E --\u003e|merge| H[\"[3, 5]\"] F \u0026 G --\u003e|merge| I[\"[1, 8]\"] H \u0026 I --\u003e|merge| J[\"[1, 3, 5, 8] ✓\"] Example 1: \\(F_n \u0026lt; \\left(\\frac{7}{4}\\right)^n\\) # This result does not follow directly from simple induction: \\(F_{k+1} = F_k + F_{k-1}\\) depends on two earlier cases, and the simple IH delivers only one. Since the two inductions are equivalent, there are ways around it (proving, for example, \\(P(k)\\) and \\(P(k+1)\\) together), but strong induction makes the proof direct.\nBase: \\(P(1)\\): \\(F_1 = 1 \u0026lt; \\dfrac{7}{4}\\) ✓\n\\(P(2)\\): \\(F_2 = 1 \u0026lt; \\left(\\dfrac{7}{4}\\right)^2 = \\dfrac{49}{16}\\) ✓\nSIH: \\(F_j \u0026lt; \\left(\\dfrac{7}{4}\\right)^j\\) for \\(j = 1, 2, \\ldots, k\\).\nInductive step (\\(n = k+1\\), with \\(k \\geq 2\\)):\n$$F_{k+1} = F_k + F_{k-1} \\stackrel{\\text{SIH}}{\u003c} \\left(\\frac{7}{4}\\right)^k + \\left(\\frac{7}{4}\\right)^{k-1} = \\left(\\frac{7}{4}\\right)^{k-1}\\left(\\frac{7}{4} + 1\\right) = \\left(\\frac{7}{4}\\right)^{k-1} \\cdot \\frac{11}{4}$$Since \\(\\dfrac{11}{4} \u0026lt; \\left(\\dfrac{7}{4}\\right)^2 = \\dfrac{49}{16}\\):\n$$F_{k+1} \u003c \\left(\\frac{7}{4}\\right)^{k-1} \\cdot \\left(\\frac{7}{4}\\right)^2 = \\left(\\frac{7}{4}\\right)^{k+1}$$Therefore, \\(F_n \u0026lt; \\left(\\dfrac{7}{4}\\right)^n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nGeneralized Strong Induction (base at \\(n_0\\)) # Just as the PMI has a generalized version, so does strong induction:\nGeneralized Strong Induction To prove \\(P(n)\\) for every \\(n \\geq n_0\\):\nBase: verify \\(P(n_0)\\) Hypothesis: \\(P(n_0), P(n_0+1), \\ldots, P(k)\\) are true Step: prove \\(P(k+1)\\) Example 2: Prime Factorization # Example 2 applies the generalized variant with \\(n_0 = 2\\): the statement holds for every \\(n \u0026gt; 1\\), and there are no integers greater than 1 and smaller than 2 to check before the base case.\nFundamental Theorem of Arithmetic (existence) Every integer \\(n \u0026gt; 1\\) is prime or can be written as a product of prime numbers.\nBase (\\(n = 2\\)): \\(2\\) is prime ✓\nSIH: \\(P(2), P(3), \\ldots, P(k)\\) are true.\nStep (\\(n = k+1\\)):\nIf \\(k+1\\) is prime, \\(P(k+1)\\) is true. If \\(k+1\\) is not prime, then \\(k+1 = a \\cdot b\\) with \\(2 \\leq a, b \u0026lt; k+1\\). By the SIH, \\(a\\) and \\(b\\) are prime or products of primes. Hence \\(k+1 = a \\cdot b\\) is also a product of primes. Therefore, every integer \\(n \u0026gt; 1\\) is prime or a product of primes, which is what we wanted to prove. \\(\\blacksquare\\)\nNote Why is strong induction the natural tool here? Because when \\(k+1\\) decomposes as \\(a \\cdot b\\), there is no guarantee that \\(a\\) or \\(b\\) is equal to \\(k\\). It can be any value between 2 and \\(k-1\\). The simple hypothesis, which delivers only \\(P(k)\\), does not support this argument.\nExample 3: The Chocolate Bar # A chocolate bar of \\(m \\times n\\) squares can be broken into individual squares in many different orders. How many breaks are needed in total?\nLet \\(Q(k)\\): \u0026ldquo;breaking a piece of \\(k\\) squares into units requires exactly \\(k-1\\) breaks.\u0026rdquo; We prove \\(Q(k)\\) by strong induction on \\(k\\).\nBase (\\(k=1\\)): a single square is already broken apart; \\(0 = 1-1\\) breaks. ✓\nSIH: \\(Q(j)\\) is true for every \\(j = 1, 2, \\ldots, k\\).\nInductive step (\\(k+1\\) squares):\nThe first break splits the piece into two parts of sizes \\(a\\) and \\(b\\), with \\(a + b = k+1\\) and \\(1 \\leq a, b \\leq k\\). By the SIH, the part with \\(a\\) squares needs \\(a-1\\) breaks and the one with \\(b\\) needs \\(b-1\\). The total is:\n$$1 + (a-1) + (b-1) = a + b - 1 = (k+1) - 1$$Therefore, a piece of \\(k\\) squares requires exactly \\(k - 1\\) breaks for every \\(k \\geq 1\\), which is what we wanted to prove. \\(\\blacksquare\\)\nIn particular, an \\(m \\times n\\) bar, with \\(mn\\) squares, needs exactly \\(mn - 1\\) breaks, whatever the order.\nNote Strong induction is the natural tool here: when breaking \\(k+1\\) squares, the sizes \\(a\\) and \\(b\\) can be any values between 1 and \\(k\\), not necessarily \\(k\\). The SIH guarantees correctness for all these cases at once.\nComparison: Simple Induction vs. Strong Induction # Feature Simple Induction Strong Induction Hypothesis \\(P(k)\\) is true \\(P(1), \\ldots, P(k)\\) are true Step \\(P(k) \\Rightarrow P(k+1)\\) \\(P(1) \\wedge \\cdots \\wedge P(k) \\Rightarrow P(k+1)\\) When to use Next step depends only on \\(k\\) Next step depends on multiple cases Equivalence Yes, they are equivalent Yes, they are equivalent Exercises # Practice by applying the structure of strong induction. Exercises 1 and 2 are of intermediate difficulty; exercises 3 and 4 require more algebraic work; exercise 5 is a fundamental result in computer science.\nExercise 1: Bound for Fibonacci Prove by strong induction that \\(F_n \\leq 2^{n-1}\\) for every \\(n \\geq 1\\).\nProof Base (\\(n=1\\)): \\(F_1 = 1 = 2^0\\) ✓\nBase (\\(n=2\\)): \\(F_2 = 1 \\leq 2^1 = 2\\) ✓\nSIH: \\(F_j \\leq 2^{j-1}\\) for every \\(j = 1, 2, \\ldots, k\\) with \\(k \\geq 2\\).\nInductive step (\\(n = k+1\\)): $$F_{k+1} = F_k + F_{k-1} \\stackrel{\\text{SIH}}{\\leq} 2^{k-1} + 2^{k-2} = 3 \\cdot 2^{k-2} \\leq 4 \\cdot 2^{k-2} = 2^k$$Therefore, \\(F_n \\leq 2^{n-1}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 2: Postage stamps Prove by strong induction that any amount \\(n \\geq 8\\) cents can be paid exactly using 3-cent and 5-cent stamps.\nProof Base (\\(n=8\\)): \\(8 = 3 + 5\\) ✓\nBase (\\(n=9\\)): \\(9 = 3 + 3 + 3\\) ✓\nBase (\\(n=10\\)): \\(10 = 5 + 5\\) ✓\nSIH: every amount \\(j\\) with \\(8 \\leq j \\leq k\\) can be paid, for some \\(k \\geq 10\\).\nInductive step (\\(n = k+1 \\geq 11\\)):\nSince \\(k+1 \\geq 11\\), we have \\((k+1) - 3 = k-2 \\geq 8\\). By the SIH, \\(k-2\\) cents can be paid with 3-cent and 5-cent stamps. Adding one 3-cent stamp, we get \\((k-2)+3 = k+1\\) cents.\nTherefore, every amount \\(n \\geq 8\\) cents can be paid, which is what we wanted to prove. \\(\\blacksquare\\)\nThe Frobenius Problem The statement starts at 8 because 7 is the largest amount that cannot be paid with 3-cent and 5-cent stamps, and this is no accident. Given two coprime positive integers \\(a\\) and \\(b\\), the largest value that cannot be written as a sum of terms \\(a\\) and \\(b\\) is \\(ab - a - b\\); for \\(a = 3\\) and \\(b = 5\\): \\(15 - 3 - 5 = 7\\). The formula already appears in a work by James Joseph Sylvester from 1882, and the general question, with any number of values, is known as the Frobenius problem, in honor of Ferdinand Frobenius. For three or more values, no explicit formula is known. For the computational complexity of a variant of the problem, see this article.\nExercise 3: Sequence with two base cases Let \\(\\{a_n\\}\\) be defined by \\(a_1 = 1\\), \\(a_2 = 5\\) and \\(a_n = a_{n-1} + 2a_{n-2}\\) for \\(n \\geq 3\\). Show by strong induction that \\(a_n = 2^n + (-1)^n\\) for every \\(n \\geq 1\\).\nProof Base (\\(n=1\\)): \\(2^1 + (-1)^1 = 1 = a_1\\) ✓\nBase (\\(n=2\\)): \\(2^2 + (-1)^2 = 5 = a_2\\) ✓\nSIH: \\(a_i = 2^i + (-1)^i\\) for every \\(i = 1, 2, \\ldots, k\\) with \\(k \\geq 2\\).\nInductive step (\\(n = k+1\\)): $$a_{k+1} = a_k + 2a_{k-1} \\stackrel{\\text{SIH}}{=} \\bigl[2^k + (-1)^k\\bigr] + 2\\bigl[2^{k-1} + (-1)^{k-1}\\bigr]$$ $$= 2^k + (-1)^k + 2^k + 2(-1)^{k-1} = 2^{k+1} + (-1)^{k-1}\\bigl[(-1) + 2\\bigr] = 2^{k+1} + (-1)^{k+1}$$Therefore, \\(a_n = 2^n + (-1)^n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nThe formula is not Binet\u0026rsquo;s The expression in Exercise 4 is named after Jacques Binet, who published it in 1843, but it was already in use in the 18th century by Abraham de Moivre, by members of the Bernoulli family and by Leonhard Euler, more than a century earlier (see the section on the formula in the Fibonacci sequence article and the history of the golden ratio). It is a classic case of Stigler\u0026rsquo;s Law: \u0026ldquo;no scientific discovery is named after its original discoverer.\u0026rdquo;\nExercise 4: Binet\u0026rsquo;s Formula for Fibonacci Binet\u0026rsquo;s Formula gives the \\(n\\)-th Fibonacci term directly: $$F_n = \\frac{1}{\\sqrt{5}}\\left(\\frac{1+\\sqrt{5}}{2}\\right)^n - \\frac{1}{\\sqrt{5}}\\left(\\frac{1-\\sqrt{5}}{2}\\right)^n$$Prove this formula by strong induction.\nProof Let \\(\\varphi = \\dfrac{1+\\sqrt{5}}{2}\\) and \\(\\psi = \\dfrac{1-\\sqrt{5}}{2}\\) (\\(\\varphi\\) is the golden ratio).\nBase (\\(n=1\\)): \\(\\dfrac{1}{\\sqrt{5}}(\\varphi - \\psi) = \\dfrac{1}{\\sqrt{5}} \\cdot \\sqrt{5} = 1 = F_1\\) ✓\nBase (\\(n=2\\)): \\(\\dfrac{1}{\\sqrt{5}}(\\varphi^2 - \\psi^2) = \\dfrac{1}{\\sqrt{5}}(\\varphi+\\psi)(\\varphi-\\psi) = \\dfrac{1}{\\sqrt{5}} \\cdot 1 \\cdot \\sqrt{5} = 1 = F_2\\) ✓\nSIH: \\(F_i = \\dfrac{\\varphi^i - \\psi^i}{\\sqrt{5}}\\) for every \\(i = 1, 2, \\ldots, k\\) with \\(k \\geq 2\\).\nInductive step (\\(n = k+1\\)): $$F_{k+1} = F_k + F_{k-1} \\stackrel{\\text{SIH}}{=} \\frac{\\varphi^k - \\psi^k}{\\sqrt{5}} + \\frac{\\varphi^{k-1} - \\psi^{k-1}}{\\sqrt{5}} = \\frac{\\varphi^{k-1}(\\varphi+1) - \\psi^{k-1}(\\psi+1)}{\\sqrt{5}}$$Since \\(\\varphi^2 = \\varphi + 1\\) and \\(\\psi^2 = \\psi + 1\\) (a property of the golden ratio): $$F_{k+1} = \\frac{\\varphi^{k-1} \\cdot \\varphi^2 - \\psi^{k-1} \\cdot \\psi^2}{\\sqrt{5}} = \\frac{\\varphi^{k+1} - \\psi^{k+1}}{\\sqrt{5}}$$Therefore, \\(F_n = \\dfrac{\\varphi^n - \\psi^n}{\\sqrt{5}}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nThe formula also explains why \\(\\varphi\\) appears: since \\(|\\psi| \u0026lt; 1\\), the term \\(\\psi^n\\) shrinks until it vanishes, and the ratio of two consecutive terms, \\(F_{n+1}/F_n\\), approaches \\(\\varphi\\). The figure below shows this approximation in the form of rectangles: squares with Fibonacci sides, fitted side by side, form rectangles ever closer to the golden proportion.\nFibonacci rectangles: the 21 × 13 rectangle has ratio 21/13 ≈ 1.615, close to φ ≈ 1.618; the arcs form the Fibonacci spiral, which approximates the golden spiral Exercise 5: Binary representation Prove by strong induction that every integer \\(n \\geq 1\\) can be written as a sum of distinct powers of 2 (binary representation).\nProof Base (\\(n=1\\)): \\(1 = 2^0\\). ✓\nSIH: every integer \\(j\\) with \\(1 \\leq j \\leq k\\) has a binary representation.\nInductive step (\\(n = k+1\\)):\nIf \\(k+1\\) is odd: then \\(k\\) is even. By the SIH, \\(k\\) has a binary representation using only the powers \\(2^1, 2^2, \\ldots\\) (since \\(k\\) is even, it does not contain \\(2^0\\)). Adding \\(2^0\\) we obtain a representation of \\(k+1\\) with distinct powers. ✓ If \\(k+1\\) is even: then \\(\\dfrac{k+1}{2} \\leq k\\). By the SIH, \\(\\dfrac{k+1}{2}\\) has a binary representation \\(2^{a_1} + \\cdots + 2^{a_r}\\). Adding 1 to each exponent (equivalent to multiplying by 2) we obtain \\(k+1 = 2^{a_1+1} + \\cdots + 2^{a_r+1}\\), with distinct powers. ✓ Therefore, every integer \\(n \\geq 1\\) can be written as a sum of distinct powers of 2, which is what we wanted to prove. \\(\\blacksquare\\)\nSummary Table # The results proved in this article, gathered for quick reference:\nResult Statement Fibonacci identity \\(F_1^2 + F_2^2 + \\cdots + F_n^2 = F_n \\cdot F_{n+1}\\) Exponential bound \\(F_n \u0026lt; \\left(\\dfrac{7}{4}\\right)^n\\) Prime factorization Every \\(n \u0026gt; 1\\) is prime or a product of primes Chocolate bar (Example 3) An \\(m \\times n\\) bar requires exactly \\(mn - 1\\) breaks Bound for Fibonacci (Ex. 1) \\(F_n \\leq 2^{n-1}\\) Postage stamps (Ex. 2) Every \\(n \\geq 8\\) cents can be paid with 3-cent and 5-cent stamps Recurrent sequence (Ex. 3) \\(a_n = 2^n + (-1)^n\\) where \\(a_n = a_{n-1} + 2a_{n-2}\\) Binet\u0026rsquo;s Formula (Ex. 4) \\(F_n = \\dfrac{\\varphi^n - \\psi^n}{\\sqrt{5}}\\) with \\(\\varphi = \\dfrac{1+\\sqrt{5}}{2}\\) Binary representation (Ex. 5) Every integer \\(n \\geq 1\\) has a base-2 representation Next Steps # The Fibonacci recurrence is an example of a recurrence relation, a formula that defines each term from earlier terms. The next article generalizes this idea, presenting systematic techniques for solving this kind of equation.\nSee you then!\n","date":"2026-06-23","externalUrl":null,"permalink":"/2026/06/strong-induction/","section":"Blog","summary":"What if the step depends on more than one earlier case, as in Fibonacci? Strong induction assumes all the earlier cases. We will see the technique at work on prime factorization, the chocolate bar, postage stamps and Binet’s formula, which in floating point fails from the 71st term on.","title":"Strong Induction: When the Step Needs More Context","type":"posts"},{"content":"","date":"2026-06-18","externalUrl":null,"permalink":"/tags/logic/","section":"tags","summary":"","title":"logic","type":"tags"},{"content":"","date":"2026-06-18","externalUrl":null,"permalink":"/tags/logica/","section":"tags","summary":"","title":"lógica","type":"tags"},{"content":" How would you prove that \\(1 + 2 + 3 + \\cdots + n = \\dfrac{n(n+1)}{2}\\) for every natural number \\(n\\)? Checking case by case is impossible: there are infinitely many naturals. We need an argument that covers all of them at once.\nThat is the job of the Principle of Mathematical Induction, one of the most elegant proof tools in mathematics.\nWhy Mathematical Induction? # Induction shows up whenever we want to state something about all natural numbers at the same time. In discrete mathematics and computer science, this happens often:\nSum and product formulas: computing the sum of a progression, the product of a sequence, or the number of subsets of a set with \\(n\\) elements requires an argument that covers every \\(n \\geq 1\\). Algorithm analysis: the complexity of nested loops, sums of geometric series and other formulas that appear in performance analysis are typically proved by induction. Correctness of recursive functions: a recursive function solves the base case directly and delegates the rest to a smaller call; proving that it returns the correct result for every \\(n\\) is, structurally, a proof by induction. Loop invariants: showing that a property holds after each iteration of a for or while also follows the logic of induction, with a base case (before the first iteration) and a step (from \\(k\\) to \\(k+1\\)). Formal language theory: properties of grammars and automata are often proved by structural induction, a generalization of PMI to recursive structures. This article presents the method and its fundamental examples. The connections with programming appear throughout the text.\nThe Intuition: Dominoes and a Staircase # Imagine an infinite row of standing dominoes. If you know that:\nThe first domino falls, and whenever a domino falls, the next one falls too, then all the dominoes will fall, even though there are infinitely many of them.\nThe figure below shows this row with each domino labeled by the statement it represents: the first one has already fallen, and each arrow indicates that the fall of one domino knocks over the next.\nThe base case knocks over the first domino; the inductive step guarantees that every domino that falls knocks over the next Another image: an infinite staircase. If you can step onto the first stair and, standing on any stair, you can always step onto the next, then you reach any stair of the staircase, however high it is. There is no need to count the stairs or climb them all at once: the two guarantees are enough.\nThe resulting chain of implications propagates indefinitely:\n\u0026#x2197; flowchart LR P1[\"P(1)\"] --\u003e P2[\"P(2)\"] P2 --\u003e P3[\"P(3)\"] P3 --\u003e P4[\"P(4)\"] P4 --\u003e PN[\"...\"] That is the logic of mathematical induction.\nThe Principle of Mathematical Induction # Principle of Mathematical Induction (PMI) Let \\(P(n)\\) be a property about \\(n \\in \\mathbb{N}\\). If:\nBase case: \\(P(1)\\) is true Inductive hypothesis (IH): assume that \\(P(k)\\) is true for an arbitrarily fixed \\(k \\geq 1\\) Inductive step: using the IH, prove that \\(P(k+1)\\) is true Then \\(P(n)\\) is true for all \\(n \\in \\mathbb{N}\\).\nPMI is a central technique for proving results involving natural numbers: sums, inequalities, divisibility and recurrent properties.\nThe idea is old. Proofs that use the method implicitly appear with the Persian mathematician al-Karaji, around the year 1000, and one of the first rigorous uses is attributed to Gersonides (Maasei Hoshev, 1321). In 1575, Francesco Maurolico used the technique to prove that the sum of the first \\(n\\) odd numbers is \\(n^2\\), the second example in this article. The first explicit formulation of the principle is by Blaise Pascal, in the Traité du triangle arithmétique, published in 1665, and the name \u0026ldquo;mathematical induction\u0026rdquo; was given by Augustus De Morgan in 1838, the same De Morgan of the laws seen in the article on operations between sets. The full history is in the Mathematical induction entry.\nIs PMI a theorem? No: it is part of the very definition of the natural numbers. In the Peano axioms, published by Giuseppe Peano in 1889, induction is one of the axioms that characterize \\(\\mathbb{N}\\), and Peano, like this series, started the naturals at 1. Whoever accepts the naturals with this structure accepts, along with them, that induction works.\nStructure of a Proof by Induction # Every proof by induction follows the same three-step structure:\nStep What to do 1. Base case Verify that \\(P(1)\\) is true 2. Inductive hypothesis (IH) Assume that \\(P(k)\\) is true for an arbitrarily fixed \\(k \\geq 1\\) 3. Inductive step Prove \\(P(k+1)\\) using the IH After verifying the base case and carrying out the inductive step, the conclusion follows automatically: \\(P(n)\\) holds for all \\(n \\in \\mathbb{N}\\).\nWorked Examples # Sum of the first \\(n\\) naturals # The story goes that Carl Friedrich Gauss, still a child, was given the task by his teacher of adding the numbers from 1 to 100 and answered 5050 within moments: he paired the first with the last (\\(1 + 100\\)), the second with the second to last (\\(2 + 99\\)), and so on, forming 50 pairs that each add up to 101. The story is probably apocryphal (see Gauss\u0026rsquo;s Day of Reckoning, in American Scientist), but the trick works for any \\(n\\) and suggests the formula; induction proves that it holds for all \\(n\\).\n$$P(n):\\ 1 + 2 + \\cdots + n = \\frac{n(n+1)}{2}$$Base case (\\(n = 1\\)): \\(1 = \\dfrac{1 \\cdot 2}{2} = 1\\) ✓\nIH: \\(1 + 2 + \\cdots + k = \\dfrac{k(k+1)}{2}\\)\nInductive step (\\(n = k+1\\)):\n$$1 + 2 + \\cdots + k + (k+1) \\stackrel{\\text{IH}}{=} \\frac{k(k+1)}{2} + (k+1) = (k+1)\\left(\\frac{k}{2} + 1\\right) = \\frac{(k+1)(k+2)}{2}$$Therefore, \\(1 + 2 + \\cdots + n = \\dfrac{n(n+1)}{2}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nApplication: induction and recursive functions The proof above has a direct correspondence with the recursive implementation of the sum in Python:\ndef sum_to(n): if n == 1: return 1 return sum_to(n - 1) + n print(sum_to(100)) # 5050 The three steps of induction map exactly to the three elements of the function:\nInduction Code Base case: \\(P(1)\\) is true if n == 1: return 1 IH: assume that sum_to(n-1) returns \\(\\frac{(n-1)n}{2}\\) recursive call sum_to(n - 1) Step: sum_to(n-1) + n produces \\(\\frac{n(n+1)}{2}\\) return sum_to(n - 1) + n Proving the formula by induction is proving that the recursive function is correct for all \\(n \\in \\mathbb{N}\\). Every well-formed recursive function implicitly carries a proof by induction. There is also a practical consequence: the recursive function and the equivalent for loop run in \\(O(n)\\), that is, the time grows with \\(n\\). The closed-form formula \\(\\frac{n(n+1)}{2}\\) runs in \\(O(1)\\), constant time. The proof by induction is the mathematical guarantee that this substitution is safe: both sides are equivalent for all \\(n\\), without exception. In the article on recurrence relations, we will see how this same function generates the equation \\(S(n) = S(n-1) + n\\) and how to solve it formally to obtain the closed-form formula.\nSum of the first \\(n\\) odd numbers # $$P(n):\\ 1 + 3 + 5 + \\cdots + (2n - 1) = n^2$$This formula has a geometric beauty: the sum of the first \\(n\\) odd numbers is always a perfect square. The figure below shows why: each new odd number is an L that completes the next square, which is exactly the step \\(k^2 + (2k+1) = (k+1)^2\\) that the proof does with algebra.\n1 + 3 + 5 + 7 + 9 = 25: each new L, in a strong color, turns the (n−1) × (n−1) square into the n × n square Base case (\\(n = 1\\)): \\(1 = 1^2\\) ✓\nIH: \\(1 + 3 + \\cdots + (2k - 1) = k^2\\)\nInductive step (\\(n = k+1\\)):\n$$1 + 3 + \\cdots + (2k-1) + (2(k+1)-1) \\stackrel{\\text{IH}}{=} k^2 + 2k + 1 = (k+1)^2$$Therefore, \\(1 + 3 + \\cdots + (2n-1) = n^2\\), which is what we wanted to prove. \\(\\blacksquare\\)\nDivisibility # $$P(n):\\ 8 \\mid (3^{2n} - 1)$$Base case (\\(n = 1\\)): \\(3^2 - 1 = 8\\), and \\(8 \\mid 8\\) ✓\nIH: \\(8 \\mid (3^{2k} - 1)\\), that is, \\(3^{2k} - 1 = 8m\\) for some \\(m \\in \\mathbb{Z}\\)\nInductive step (\\(n = k+1\\)):\nWe want to show that \\(8 \\mid (3^{2(k+1)} - 1)\\). First, we expand the exponent:\n$$3^{2(k+1)} - 1 = 3^{2k+2} - 1 = 3^2 \\cdot 3^{2k} - 1 = 9 \\cdot 3^{2k} - 1$$Now we rewrite it to isolate \\(3^{2k} - 1\\), which appears in the IH:\n$$9 \\cdot 3^{2k} - 1 = 9 \\cdot 3^{2k} - 9 + 8 = 9(3^{2k} - 1) + 8$$By the IH, \\(3^{2k} - 1 = 8m\\) for some \\(m \\in \\mathbb{Z}\\), so:\n$$9(3^{2k} - 1) + 8 = 9 \\cdot 8m + 8 = 8(9m + 1)$$Therefore, \\(8 \\mid (3^{2n} - 1)\\), which is what we wanted to prove. \\(\\blacksquare\\)\nApplication: loop invariants The same logic of induction guarantees the correctness of iterative loops. Consider the for below, which computes the running sum:\nn = 100 total = 0 for k in range(1, n + 1): total += k print(total == n * (n + 1) // 2) # True Define the invariant \\(P(k)\\): \u0026ldquo;after the \\(k\\)-th iteration, total == k*(k+1)//2\u0026rdquo;. Then:\nBase case (\\(k = 0\\), before the loop): total = 0 and \\(0 \\cdot 1 / 2 = 0\\) ✓ Step (from \\(k\\) to \\(k+1\\)): if total == k*(k+1)//2 before iteration \\(k+1\\), then after total += k+1 we have \\(\\frac{k(k+1)}{2} + (k+1) = \\frac{(k+1)(k+2)}{2}\\) ✓ At the end of the loop, \\(k = n\\), so total == n*(n+1)//2. This argument is induction in disguise, and it is the mechanism that formal verifiers (such as those based on Hoare logic) use to prove the correctness of iterative programs.\nGeneralized PMI: Base Case at \\(n_0 \\neq 1\\) # Not every property holds from \\(n = 1\\) on. When we want to prove \\(P(n)\\) for all \\(n \\geq n_0\\), we use the generalized version:\nGeneralized PMI To prove \\(P(n)\\) for all \\(n \\geq n_0\\):\nBase case: verify that \\(P(n_0)\\) is true Inductive hypothesis (IH): assume that \\(P(k)\\) is true for an arbitrarily fixed \\(k \\geq n_0\\) Inductive step: using the IH, prove that \\(P(k+1)\\) is true When \\(n_0 = 1\\), we get the usual PMI as a special case.\nExample: \\(n^2 \u0026gt; 3n\\) for \\(n \\geq 4\\) # Why not start from \\(n=1\\)? The property \\(n^2 \u0026gt; 3n\\) does not hold for \\(n = 1, 2, 3\\):\n\\(1^2 = 1 \\not\u0026gt; 3\\) \\(2^2 = 4 \\not\u0026gt; 6\\) \\(3^2 = 9 \\not\u0026gt; 9\\) Induction can only start at \\(n_0 = 4\\).\nBase case (\\(n = 4\\)): \\(16 \u0026gt; 12\\) ✓\nIH: \\(k^2 \u0026gt; 3k\\) for some \\(k \\geq 4\\)\nInductive step (\\(n = k+1\\)):\n$$(k+1)^2 = k^2 + 2k + 1 \\stackrel{\\text{IH}}{\u003e} 3k + 2k + 1 = 5k + 1 \u003e 3(k+1) = 3k + 3$$(valid because \\(5k + 1 \u0026gt; 3k + 3 \\iff 2k \u0026gt; 2 \\iff k \u0026gt; 1\\), which is true for \\(k \\geq 4\\))\nTherefore, \\(n^2 \u0026gt; 3n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nWhen Induction Fails # The method is only as reliable as each of its parts. A famous example shows what happens when the inductive step has a hole: in a satirical article from 1961, the mathematician Joel E. Cohen \u0026ldquo;proved\u0026rdquo; that all horses are the same color.\n$$P(n):\\ \\text{in any group of } n \\text{ horses, all have the same color}$$Base case (\\(n = 1\\)): a group with a single horse has a single color ✓\nIH: in any group of \\(k\\) horses, all have the same color.\n\u0026ldquo;Inductive step\u0026rdquo; (\\(n = k+1\\)): number the horses of a group of \\(k+1\\) from 1 to \\(k+1\\). Without the last one, horses 1 to \\(k\\) remain, which by the IH have the same color. Without the first one, horses 2 to \\(k+1\\) remain, which by the IH also have the same color. Since the two groups have horses in common, the two colors are one and the same, and all \\(k+1\\) horses have the same color.\nWhere is the error? The base case is right, and the argument of the step works for \\(k \\geq 2\\). The hole is at \\(k = 1\\): in a group of 2 horses, \u0026ldquo;horses 1 to \\(k\\)\u0026rdquo; is just horse 1 and \u0026ldquo;those from 2 to \\(k+1\\)\u0026rdquo; is just horse 2. There is no horse in common, and nothing connects the two colors. The implication \\(P(1) \\Rightarrow P(2)\\) was never proved: the first domino falls, but nothing guarantees that it knocks over the second. The inductive step must hold for every \\(k \\geq n_0\\), including the first one, precisely where this argument breaks down. More at All horses are the same color.\nExercises # Practice by applying the three-step structure seen above. The proofs are available for checking after you try.\nExercise 1: Sum of powers of 2 Prove by induction that \\(1 + 2 + 4 + \\cdots + 2^{n-1} = 2^n - 1\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(1 = 2^1 - 1\\) ✓\nIH: \\(1 + 2 + \\cdots + 2^{k-1} = 2^k - 1\\)\nInductive step: $$1 + 2 + \\cdots + 2^{k-1} + 2^k \\stackrel{\\text{IH}}{=} (2^k - 1) + 2^k = 2 \\cdot 2^k - 1 = 2^{k+1} - 1$$Therefore, \\(1 + 2 + \\cdots + 2^{n-1} = 2^n - 1\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 2: Sum of squares Prove that \\(1^2 + 2^2 + \\cdots + n^2 = \\dfrac{n(n+1)(2n+1)}{6}\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(1 = \\dfrac{1 \\cdot 2 \\cdot 3}{6} = 1\\) ✓\nIH: \\(1^2 + 2^2 + \\cdots + k^2 = \\dfrac{k(k+1)(2k+1)}{6}\\)\nInductive step: $$1^2 + \\cdots + k^2 + (k+1)^2 \\stackrel{\\text{IH}}{=} \\frac{k(k+1)(2k+1)}{6} + (k+1)^2 = (k+1)\\left(\\frac{k(2k+1)}{6} + (k+1)\\right)$$$$= (k+1) \\cdot \\frac{2k^2+7k+6}{6} = \\frac{(k+1)(k+2)(2k+3)}{6}$$Therefore, \\(1^2 + 2^2 + \\cdots + n^2 = \\dfrac{n(n+1)(2n+1)}{6}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 3: Sum of consecutive products Prove that \\(\\displaystyle\\sum_{i=1}^{n} i(i+1) = \\dfrac{n(n+1)(n+2)}{3}\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(1 \\cdot 2 = 2\\) and \\(\\dfrac{1 \\cdot 2 \\cdot 3}{3} = 2\\) ✓\nIH: \\(\\displaystyle\\sum_{i=1}^{k} i(i+1) = \\dfrac{k(k+1)(k+2)}{3}\\)\nInductive step: $$\\sum_{i=1}^{k+1} i(i+1) \\stackrel{\\text{IH}}{=} \\frac{k(k+1)(k+2)}{3} + (k+1)(k+2) = (k+1)(k+2)\\left(\\frac{k}{3}+1\\right) = \\frac{(k+1)(k+2)(k+3)}{3}$$Therefore, \\(\\displaystyle\\sum_{i=1}^{n} i(i+1) = \\dfrac{n(n+1)(n+2)}{3}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 4: Arithmetic progression Prove that \\(2 + 5 + 8 + \\cdots + (3n-1) = \\dfrac{n(1+3n)}{2}\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(3(1)-1 = 2\\) and \\(\\dfrac{1 \\cdot 4}{2} = 2\\) ✓\nIH: \\(\\displaystyle\\sum_{i=1}^{k}(3i-1) = \\dfrac{k(1+3k)}{2}\\)\nInductive step: $$\\sum_{i=1}^{k+1}(3i-1) \\stackrel{\\text{IH}}{=} \\frac{k(1+3k)}{2} + (3k+2) = \\frac{k+3k^2+6k+4}{2} = \\frac{3k^2+7k+4}{2} = \\frac{(k+1)(3k+4)}{2}$$Therefore, \\(2 + 5 + \\cdots + (3n-1) = \\dfrac{n(1+3n)}{2}\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 5: Telescoping product Prove that \\(\\left(1+1\\right)\\left(1+\\dfrac{1}{2}\\right)\\left(1+\\dfrac{1}{3}\\right)\\cdots\\left(1+\\dfrac{1}{n}\\right) = n+1\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(1 + \\dfrac{1}{1} = 2 = 1+1\\) ✓\nIH: \\(\\displaystyle\\prod_{i=1}^{k}\\left(1+\\dfrac{1}{i}\\right) = k+1\\)\nInductive step: $$\\prod_{i=1}^{k+1}\\left(1+\\frac{1}{i}\\right) \\stackrel{\\text{IH}}{=} (k+1)\\left(1+\\frac{1}{k+1}\\right) = (k+1)\\cdot\\frac{k+2}{k+1} = k+2$$Therefore, \\(\\displaystyle\\prod_{i=1}^{n}\\left(1+\\dfrac{1}{i}\\right) = n+1\\), which is what we wanted to prove. \\(\\blacksquare\\)\nExercise 6: Divisibility Prove that \\(2 \\mid n^2 + n\\) for all \\(n \\in \\mathbb{N}\\).\nProof Base case (\\(n=1\\)): \\(1+1 = 2\\), divisible by 2 ✓\nIH: \\(k^2 + k = 2q\\) for some \\(q \\in \\mathbb{N}\\)\nInductive step: $$(k+1)^2 + (k+1) = k^2 + 2k + 1 + k + 1 = \\underbrace{(k^2+k)}_{2q} + 2(k+1) = 2\\bigl(q + k + 1\\bigr)$$Therefore, \\(2 \\mid n^2 + n\\), which is what we wanted to prove. \\(\\blacksquare\\)\nRemark: The factorization \\(n^2 + n = n(n+1)\\) reveals the intuition (a product of consecutive integers, one of them always even), but it is induction that formalizes the proof for all naturals.\nSummary Table # The results proved in this article, gathered for quick reference:\nResult Statement Sum of the naturals \\(1+2+\\cdots+n = \\dfrac{n(n+1)}{2}\\) Sum of the odd numbers \\(1+3+\\cdots+(2n-1) = n^2\\) Divisibility \\(8 \\mid (3^{2n}-1)\\) Inequality (\\(n \\geq 4\\)) \\(n^2 \u0026gt; 3n\\) Powers of 2 \\(1+2+\\cdots+2^{n-1} = 2^n - 1\\) Sum of squares \\(1^2+2^2+\\cdots+n^2 = \\dfrac{n(n+1)(2n+1)}{6}\\) Sum of consecutive products \\(\\displaystyle\\sum_{i=1}^{n} i(i+1) = \\dfrac{n(n+1)(n+2)}{3}\\) Arithmetic progression \\(2+5+\\cdots+(3n-1) = \\dfrac{n(1+3n)}{2}\\) Telescoping product \\(\\displaystyle\\prod_{i=1}^{n}\\left(1+\\frac{1}{i}\\right) = n+1\\) Simple divisibility \\(2 \\mid n^2+n\\) Next Steps # Standard PMI uses only the case \\(P(k)\\) to prove \\(P(k+1)\\). But some problems require all the previous cases at once. Consider, for example, proving that every integer \\(n \\geq 2\\) has a prime factor: to decompose \\(n\\), it may be necessary to invoke the hypothesis for values smaller than \\(k\\), not just for \\(k\\). Standard PMI does not reach that argument, and for that there is Strong Induction, the topic of the next article.\n","date":"2026-06-18","externalUrl":null,"permalink":"/2026/06/mathematical-induction/","section":"Blog","summary":"If the first domino falls and each domino knocks over the next, they all fall. That is the logic of the Principle of Mathematical Induction. We will see the structure of the proof (base case, hypothesis and step) in sums, divisibility and inequalities, and the false proof that all horses are the same color.","title":"Mathematical Induction: Proving Properties for All Natural Numbers","type":"posts"},{"content":" How would you prove that \\(1 + 2 + \\cdots + n = \\dfrac{n(n+1)}{2}\\) holds for every natural number? Checking case by case is impossible: there are infinitely many naturals. We need a different argument, one that covers every case at once.\nThat is the central question of this series, and the answer is the Principle of Mathematical Induction.\nWhat you will learn # The article Mathematical Induction: Proving Properties for All Naturals presents the classic structure of a proof by induction (base case, inductive hypothesis and step) with complete examples of sums, products and divisibility. It also covers the generalized version with base \\(n_0 \\neq 1\\).\nThe article Strong Induction: When the Step Needs More Context extends the classic method to situations where step \\(k+1\\) depends on several previous cases. The Fibonacci sequence, the proof of prime factorization and Binet\u0026rsquo;s formula show when and why the stronger hypothesis is needed.\nFinally, the article Recurrence Relations: Defining Sequences by Themselves generalizes the idea of recurrence: how to set up the equation that describes a sequence and how to solve it by substitution to get closed-form formulas. The Tower of Hanoi, valid arithmetic expressions and the ovals in the plane problem are the central examples. The article also presents the characteristic equation, a systematic technique for solving second-order recurrences (those in which each term depends on the two previous ones, like Fibonacci), and divide-and-conquer recurrences, with the Master Theorem.\nThe diagram below shows how the three articles build on each other: strong induction starts from mathematical induction, and recurrence relations use both to verify the closed-form formulas they find.\n\u0026#x2197; flowchart LR A[\"Mathematical Induction\nbase, hypothesis and step\"] --\u003e|\"stronger hypothesis\"| B[\"Strong Induction\nall previous cases\"] A --\u003e|\"verifies the\nclosed-form formulas\"| C[\"Recurrence Relations\nsubstitution, characteristic\nequation and\ndivide and conquer\"] B --\u003e|\"verifies the\nsecond-order ones\"| C Why induction and recurrence? # Mathematical induction and recurrence relations show up in practically all of the rest of discrete mathematics:\nAlgorithm analysis: the complexity of MergeSort satisfies \\(T(n) = 2T(n/2) + n\\), a recurrence that shows the algorithm is \\(O(n \\log n)\\), as the section on divide and conquer of the article on recurrence relations shows. Combinatorics: the combination formulas and the coefficients of the binomial theorem have natural proofs by induction. Graphs: properties of trees (number of edges, paths) are proved by induction on the number of vertices. Number theory: divisibility, prime factorization and the very structure of the naturals depend on the inductive principle. Mastering these tools is essential to reason precisely about discrete structures.\nPrerequisites # Familiarity with natural numbers, basic algebra and summation notation (\\(\\sum\\)). The previous series, The Language of Sets, covers the foundations of set theory, but it is not a strict prerequisite for this material.\nIf you want to understand the overall plan of the Discrete Mathematics series on the site (which topics will be covered, why they were chosen and the motivation behind them), the article Discrete Mathematics: Introduction and Roadmap lays out the complete map of the four planned series.\nSee you next time!\n","date":"2026-06-17","externalUrl":null,"permalink":"/2026/06/from-base-case-to-infinity/","section":"Blog","summary":"How do you prove a formula holds for every natural number, if there are infinitely many of them? With mathematical induction. This introduction maps out the series: classic induction, strong induction and recurrence relations, from Fibonacci to the Tower of Hanoi.","title":"From Base Case to Infinity: An Introduction to Induction and Recurrence","type":"posts"},{"content":"Uma introdução à Teoria dos Conjuntos como base da Matemática Discreta: definições, operações (união, interseção, complemento), diagramas de Venn, Princípio da Inclusão-Exclusão e cardinalidade — com exemplos práticos em Python e SQL.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-06-16","externalUrl":null,"permalink":"/series/a-linguagem-dos-conjuntos/","section":"Séries","summary":"","title":"A Linguagem dos Conjuntos","type":"series"},{"content":"","date":"2026-06-16","externalUrl":null,"permalink":"/tags/conjuntos/","section":"tags","summary":"","title":"conjuntos","type":"tags"},{"content":"","date":"2026-06-16","externalUrl":null,"permalink":"/tags/sets/","section":"tags","summary":"","title":"sets","type":"tags"},{"content":" The previous articles in this series presented set theory in mathematical language: what a set is, how Venn diagrams and operations work, and the notions of cardinality. Now it is time to see that theory in action in Python.\nThe good news: you do not need to import any special library. Python implements mathematical sets natively with the set type, mapping directly onto the concepts we have already studied, including the notation for membership, inclusion, and the operations of union, intersection, and difference.\nThe set type # In Python, a set is created with braces {} separating the elements, or with the set() constructor receiving any iterable:\nA = {1, 2, 3} B = set([3, 1, 2]) print(A) # {1, 2, 3} print(A == B) # True Two mathematical properties apply automatically:\nUniqueness: repeated elements are silently ignored:\nC = {1, 2, 2, 3, 3, 3} print(C) # {1, 2, 3} print(len(C)) # 3 No order: Python does not guarantee any iteration order, just as \\(\\{1,2,3\\} = \\{3,2,1\\}\\) mathematically:\nprint({1, 2, 3} == {3, 2, 1}) # True For this reason, in the output comments of this article, the elements of a set appear in an order chosen for readability (numbers in ascending order). When you run the code, Python may display them in another order: the set is the same.\nThe empty set: set(), not {} # Warning {} creates an empty dictionary, not an empty set. To create \\(\\emptyset\\) in Python, always use set():\nempty = set() dictionary = {} print(type(empty)) # \u0026lt;class \u0026#39;set\u0026#39;\u0026gt; print(type(dictionary)) # \u0026lt;class \u0026#39;dict\u0026#39;\u0026gt; Set comprehension notation # The mathematical set-builder definition \\(C = \\{x \\in \\mathbb{N} \\mid x \\text{ is even and } x \u0026lt; 10\\}\\) has an almost literal translation in Python:\nC = {x for x in range(1, 10) if x % 2 == 0} print(C) # {2, 4, 6, 8} The structure {expression for variable in iterable if condition} is called a set comprehension and follows exactly the logic of mathematical set-builder notation.\nExample: powers of 2 less than 100 powers = {2**n for n in range(10) if 2**n \u0026lt; 100} print(powers) # {1, 2, 4, 8, 16, 32, 64} Equivalent to \\(\\{2^n \\mid n \\in \\{0,1,\\ldots,9\\},\\ 2^n \u0026lt; 100\\}\\).\nMembership and inclusion # Python offers operators that map directly onto the mathematical symbols:\nMathematical concept Symbol Python Membership \\(x \\in A\\) x in A Non-membership \\(x \\notin A\\) x not in A Equality \\(A = B\\) A == B Subset \\(A \\subseteq B\\) A \u0026lt;= B or A.issubset(B) Proper subset \\(A \\subset B\\) A \u0026lt; B Superset \\(B \\supseteq A\\) B \u0026gt;= A or B.issuperset(A) Proper superset \\(B \\supset A\\) B \u0026gt; A naturals = {1, 2, 3, 4, 5} evens = {2, 4} print(3 in naturals) # True — 3 ∈ naturals print(6 not in naturals) # True — 6 ∉ naturals print(evens \u0026lt;= naturals) # True — evens ⊆ naturals print(evens \u0026lt; naturals) # True — evens ⊂ naturals (proper subset) print(naturals \u0026gt;= evens) # True — naturals ⊇ evens print(set() \u0026lt;= naturals) # True — ∅ ⊆ A for any A The last line confirms in code the property \\(\\emptyset \\subseteq A\\) for any set \\(A\\).\n\u0026lt;= vs issubset(): what is the difference? issubset() accepts any iterable as an argument, while \u0026lt;= requires both sides to be a set. Use issubset() when comparing with lists or tuples:\n{1, 2}.issubset([1, 2, 3]) # True — works with a list {1, 2} \u0026lt;= [1, 2, 3] # TypeError — requires a set Operations between sets # The four fundamental operations have dedicated operators:\nOperation Symbol Python Union \\(A \\cup B\\) A | B or A.union(B) Intersection \\(A \\cap B\\) A \u0026amp; B or A.intersection(B) Difference \\(A - B\\) A - B or A.difference(B) Symmetric difference \\(A \\mathbin{\\Delta} B\\) A ^ B or A.symmetric_difference(B) A = {1, 2, 3, 4} B = {3, 4, 5, 6} print(A | B) # {1, 2, 3, 4, 5, 6} — A ∪ B print(A \u0026amp; B) # {3, 4} — A ∩ B print(A - B) # {1, 2} — A − B print(B - A) # {5, 6} — B − A print(A ^ B) # {1, 2, 5, 6} — A Δ B Note The operators (|, \u0026amp;, -, ^) require both operands to be a set. The method versions (union(), intersection(), etc.) accept any iterable, which is useful when working with lists or tuples:\n{1, 2, 3}.union([3, 4, 5]) # {1, 2, 3, 4, 5} — works {1, 2, 3} | [3, 4, 5] # TypeError — requires a set In-place operators (|=, \u0026amp;=, -=, ^=) The set has assignment operators that modify the set in place. The immutable frozenset does not have them:\nA = {1, 2, 3} A |= {4, 5} # equivalent to A.update({4, 5}) print(A) # {1, 2, 3, 4, 5} A \u0026amp;= {2, 3, 4} # equivalent to A.intersection_update({2, 3, 4}) print(A) # {2, 3, 4} The same holds for -= (difference_update) and ^= (symmetric_difference_update).\nDisjointness # To check whether two sets are disjoint (\\(A \\cap B = \\emptyset\\)):\nprint({1, 2}.isdisjoint({3, 4})) # True print({1, 2}.isdisjoint({2, 3})) # False frozenset: the immutable set # Python offers a second kind of set: frozenset. The difference is that frozenset is immutable: its elements cannot be added or removed after creation.\nfs = frozenset({1, 2, 3}) # fs.add(4) # AttributeError: \u0026#39;frozenset\u0026#39; object has no attribute \u0026#39;add\u0026#39; Why does this matter? Because a frozenset can be an element of another set, something impossible with a mutable set:\n# Attempt with a mutable set {{1, 2}} # TypeError: unhashable type: \u0026#39;set\u0026#39; # With frozenset it works {frozenset({1, 2}), frozenset({3})} # OK This allows us to represent the power set \\(\\mathcal{P}(A)\\), whose elements are sets (a concept introduced in the first article of this series, where the Hasse diagram and the decision tree illustrate its structure):\nfrom itertools import combinations A = {1, 2, 3} power_set = {frozenset(sub) for r in range(len(A) + 1) for sub in combinations(A, r)} print(len(power_set)) # 8 = 2³ ✓ print(power_set) # {frozenset(), frozenset({1}), frozenset({2}), frozenset({3}), # frozenset({1, 2}), frozenset({1, 3}), frozenset({2, 3}), # frozenset({1, 2, 3})} The construction is a set comprehension, like those in the Set comprehension notation section, only with two for clauses. They read in the order they appear, like nested loops: the first goes through the sizes r from 0 to 3; the second, for each r, goes through the combinations of r elements of A generated by combinations(A, r).\nCardinality with len() # The cardinality \\(|A|\\) of a set is obtained with len():\nA = {1, 2, 3} print(len(A)) # 3 — |A| = 3 print(2 ** len(A)) # 8 — |𝒫(A)| = 2³ = 8 Why set is so fast: hashability # The x in A operation on a set has constant cost on average (\\(O(1)\\)), regardless of the size of the set. In a list, the same test requires going through the elements one by one: \\(O(n)\\).\nO(1) on average, not in the worst case The \\(O(1)\\) complexity holds on average (in the expected case): Python computes the hash of the sought value and goes straight to the corresponding position in the internal table. If many elements produce the same hash (a collision), the lookup in that slot can degrade to \\(O(n)\\). In practice, with common types (integers, strings, tuples), collisions are rare and \\(O(1)\\) is the right reference for algorithm analysis.\nDo not confuse this with amortized cost, which is the average over a sequence of operations, like the list append, which from time to time needs to reallocate everything. Here each lookup, taken on its own, is \\(O(1)\\) in the expected case and \\(O(n)\\) in the worst case.\nimport time big_set = set(range(1_000_000)) big_list = list(range(1_000_000)) start = time.perf_counter() _ = 999_999 in big_set print(f\u0026#34;set: {time.perf_counter() - start:.2e} s\u0026#34;) # ~1e-05 s start = time.perf_counter() _ = 999_999 in big_list print(f\u0026#34;list: {time.perf_counter() - start:.2e} s\u0026#34;) # ~1e-02 s The exact times vary from machine to machine; what holds steady is the difference of about three orders of magnitude, and it grows along with the size of the collection.\nThis is possible because Python computes a hash (a numeric fingerprint) of each element when inserting it. On lookup, it computes the hash of the sought value and goes straight to the position, without traversing anything.\nThe connection to the theory: in mathematics, elements only need to be well defined. However, for Python to implement this with the speed we saw above, the language imposes a practical restriction: the element must be immutable (hashable). That is why:\n{[1, 2]} # TypeError: unhashable type: \u0026#39;list\u0026#39; {(1, 2)} # OK — a tuple is immutable {\u0026#34;text\u0026#34;} # OK — a string is immutable Lists are mutable: their contents can change at any moment, which would break the hash invariant. Strings and numbers are immutable and therefore hashable and accepted as elements. So are tuples, as long as all of their elements are hashable: {(1, [2])} raises TypeError.\nThe same mechanism explains the cost of operations between sets. The intersection traverses the smaller of the two sets and tests each element in the other; the difference traverses \\(A\\) and tests each element in \\(B\\); the union has to copy both. Since each test costs \\(O(1)\\) on average, the total cost is proportional to the number of elements traversed:\nOperation Python Average cost Membership x in A \\(O(1)\\) Union A | B \\(O(\\lvert A \\rvert + \\lvert B \\rvert)\\) Intersection A \u0026amp; B \\(O(\\min(\\lvert A \\rvert, \\lvert B \\rvert))\\) Difference A - B \\(O(\\lvert A \\rvert)\\) The values are those from the TimeComplexity page of the official Python wiki, which also lists the worst case of each operation.\nFrom identities to code # The three previous articles established results that, seen as equations, may look abstract. With set in hand, it is possible to run them directly, which serves both to consolidate intuition and to test conjectures before proving them formally.\nComplement # The article on operations defines the complement of \\(A\\) as \\(\\bar{A} = U - A\\): everything that is in the universe but not in \\(A\\). Python does not offer a native complement operator, and the reason is straightforward: a set does not carry the universe with it. You have to define it explicitly:\nU = set(range(1, 11)) # universe: {1, 2, ..., 10} A = {1, 3, 5, 7, 9} # odd numbers complement_A = U - A print(complement_A) # {2, 4, 6, 8, 10} The limitation has nothing to do with hashing: a set stores only its own elements, with no reference to the larger context it sits in, and a universe such as \\(\\mathbb{N}\\) would not even fit in memory. In mathematics, the universe \\(U\\) can stay implicit; in code, it has to be explicit.\nDe Morgan\u0026rsquo;s laws # The same article proves the two identities:\n$$\\overline{A \\cup B} = \\bar{A} \\cap \\bar{B} \\qquad \\overline{A \\cap B} = \\bar{A} \\cup \\bar{B}$$With the universe already defined, both can be checked directly:\nU = set(range(1, 11)) A = {1, 2, 3, 4, 5} B = {4, 5, 6, 7, 8} complement_A = U - A complement_B = U - B # First law: complement of the union = intersection of the complements print((U - (A | B)) == (complement_A \u0026amp; complement_B)) # True # Second law: complement of the intersection = union of the complements print((U - (A \u0026amp; B)) == (complement_A | complement_B)) # True Note that the cited article presented De Morgan in the context of Boolean logic (not (A or B) ↔ not A and not B). Here the same laws appear in their set-theoretic version, and they are the same algebraic structure seen from two angles.\nInclusion-Exclusion Principle # The article on cardinality shows that, when two sets overlap, adding their cardinalities counts the intersection twice. The correction is:\n$$|A \\cup B| = |A| + |B| - |A \\cap B|$$In Python, len() and the set operators let us confirm this directly:\nA = {1, 2, 3, 4} B = {3, 4, 5, 6} pie = len(A) + len(B) - len(A \u0026amp; B) print(pie == len(A | B)) # True — 4 + 4 - 2 = 6 The equality holds for any pair of finite sets, and Python can serve as a quick checker when working with concrete examples before a formal proof.\nPractical applications # List deduplication # with_duplicates = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3] without_duplicates = list(set(with_duplicates)) print(without_duplicates) # [1, 2, 3, 4, 5, 6, 9] (order not guaranteed) Preserving order when removing duplicates If the original order matters, use dict.fromkeys() (Python 3.7+):\nwith_duplicates = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3] without_duplicates_ordered = list(dict.fromkeys(with_duplicates)) print(without_duplicates_ordered) # [3, 1, 4, 5, 9, 2, 6] (order preserved) Permission checking # required_permissions = {\u0026#34;read\u0026#34;, \u0026#34;write\u0026#34;} user_permissions = {\u0026#34;read\u0026#34;, \u0026#34;write\u0026#34;, \u0026#34;execute\u0026#34;} has_access = required_permissions \u0026lt;= user_permissions print(has_access) # True — required_permissions ⊆ user_permissions Elements in common between groups # class_a = {\u0026#34;Ana\u0026#34;, \u0026#34;Bruno\u0026#34;, \u0026#34;Carla\u0026#34;, \u0026#34;Diego\u0026#34;} class_b = {\u0026#34;Bruno\u0026#34;, \u0026#34;Carla\u0026#34;, \u0026#34;Eduardo\u0026#34;, \u0026#34;Fernanda\u0026#34;} in_both = class_a \u0026amp; class_b print(in_both) # {\u0026#39;Bruno\u0026#39;, \u0026#39;Carla\u0026#39;} only_in_a = class_a - class_b print(only_in_a) # {\u0026#39;Ana\u0026#39;, \u0026#39;Diego\u0026#39;} When not to use set # Situation Suitable type Element order matters list Repeated elements matter (multiset) collections.Counter Key → value association dict Mutable elements (lists inside) list of list Immutable set (element of another set) frozenset The flowchart below summarizes the decision of which type to use when the collection needs unique elements:\n\u0026#x2197; flowchart TD A[\"I need a collection\"] --\u003e B{\"Unique elements\nwithout order?\"} B -- \"No\" --\u003e C{\"Does order\nmatter?\"} C -- \"Yes\" --\u003e D[\"list\"] C -- \"No\" --\u003e E{\"Does multiplicity\nmatter?\"} E -- \"Yes\" --\u003e F[\"collections.Counter\"] E -- \"No\" --\u003e G{\"Key → value\nassociation?\"} G -- \"Yes\" --\u003e H[\"dict\"] G -- \"No\" --\u003e D B -- \"Yes\" --\u003e I{\"Must be\nimmutable / hashable?\"} I -- \"No\" --\u003e J[\"set\"] I -- \"Yes\" --\u003e K[\"frozenset\"] Exercises # Exercise 1: Creation and uniqueness What is the result of len({1, 1, 2, 2, 3})? And of {1, 2, 3} == {3, 2, 1}?\nSolution len({1, 1, 2, 2, 3}) → 3 (repetitions ignored) {1, 2, 3} == {3, 2, 1} → True (sets have no order) Exercise 2: Empty set What is the difference between set() and {}? Check with type().\nSolution print(type(set())) # \u0026lt;class \u0026#39;set\u0026#39;\u0026gt; print(type({})) # \u0026lt;class \u0026#39;dict\u0026#39;\u0026gt; {} creates an empty dictionary. For the mathematical empty set, use set().\nExercise 3: Inclusion Given A = {2, 4} and B = {1, 2, 3, 4, 5}, write Python expressions that check: (i) \\(A \\subseteq B\\), (ii) \\(A \\subset B\\), (iii) \\(\\emptyset \\subseteq A\\).\nSolution A, B = {2, 4}, {1, 2, 3, 4, 5} print(A \u0026lt;= B) # True — (i) A ⊆ B print(A \u0026lt; B) # True — (ii) A ⊂ B (proper subset) print(set() \u0026lt;= A) # True — (iii) ∅ ⊆ any set Exercise 4: Operations Given A = {1, 2, 3, 4} and B = {3, 4, 5, 6}, compute in Python: (i) \\(A \\cup B\\), (ii) \\(A \\cap B\\), (iii) \\(A - B\\), (iv) \\(A \\mathbin{\\Delta} B\\).\nSolution A, B = {1, 2, 3, 4}, {3, 4, 5, 6} print(A | B) # {1, 2, 3, 4, 5, 6} print(A \u0026amp; B) # {3, 4} print(A - B) # {1, 2} print(A ^ B) # {1, 2, 5, 6} Exercise 5: Practical application Given the tag lists of two articles, determine: (i) tags in common, (ii) tags exclusive to the first, (iii) whether the second covers all the tags of the first.\ntags1 = [\u0026#34;python\u0026#34;, \u0026#34;sets\u0026#34;, \u0026#34;math\u0026#34;, \u0026#34;programming\u0026#34;] tags2 = [\u0026#34;python\u0026#34;, \u0026#34;sets\u0026#34;, \u0026#34;graphs\u0026#34;, \u0026#34;programming\u0026#34;] Solution tags1 = [\u0026#34;python\u0026#34;, \u0026#34;sets\u0026#34;, \u0026#34;math\u0026#34;, \u0026#34;programming\u0026#34;] tags2 = [\u0026#34;python\u0026#34;, \u0026#34;sets\u0026#34;, \u0026#34;graphs\u0026#34;, \u0026#34;programming\u0026#34;] t1, t2 = set(tags1), set(tags2) print(t1 \u0026amp; t2) # {\u0026#39;python\u0026#39;, \u0026#39;sets\u0026#39;, \u0026#39;programming\u0026#39;} print(t1 - t2) # {\u0026#39;math\u0026#39;} print(t1 \u0026lt;= t2) # False — \u0026#39;math\u0026#39; is not in t2 Exercise 6: frozenset and the power set Build \\(\\mathcal{P}(A)\\) for \\(A = \\{1, 2, 3\\}\\) using frozenset and itertools.combinations. Verify that \\(|\\mathcal{P}(A)| = 2^3 = 8\\).\nSolution from itertools import combinations A = {1, 2, 3} power_set = {frozenset(sub) for r in range(len(A) + 1) for sub in combinations(A, r)} print(len(power_set)) # 8 ✓ Next steps # With the theory and the Python implementation consolidated, the series moves on to other topics in Discrete Mathematics. To go deeper into the use of set and frozenset, see the official documentation of the set type and the itertools module for combinatorial operations on sets.\nSee you in the next article!\n","date":"2026-06-16","externalUrl":null,"permalink":"/2026/06/python-sets/","section":"Blog","summary":"Testing whether an element is in a set takes constant time on average, while in a list the time grows with its size. We will see how set theory becomes Python code: set and frozenset, comprehension, membership and inclusion, union, intersection and difference, hashing, and practical applications.","title":"Sets in Python: set, frozenset, and Operations","type":"posts"},{"content":"An introduction to Set Theory as the foundation of Discrete Mathematics: definitions, operations (union, intersection, complement), Venn diagrams, the Inclusion-Exclusion Principle and cardinality, with practical examples in Python and SQL.\nThe articles are listed below, in the order they were written.\n","date":"2026-06-16","externalUrl":null,"permalink":"/series/the-language-of-sets/","section":"Series","summary":"","title":"The Language of Sets","type":"series"},{"content":" In the previous two articles we defined what a set is and how to operate on it. Now we reach a natural question: how many elements does a set have? This simple question hides an important subtlety: when two sets overlap and we want to count their union, a plain \\(n(A) + n(B)\\) counts some elements twice. The Inclusion-Exclusion Principle fixes this and is the first combinatorial result of this series.\nCardinality # Definition The cardinality of a set \\(A\\), denoted \\(n(A)\\) or \\(|A|\\), is the number of elements of \\(A\\).\nAbout the notation This article uses \\(n(A)\\) to avoid ambiguity with the absolute value \\(|x|\\), which appears in the examples below. In the Computer Science literature, however, the notation \\(|A|\\) is the most common: you will find it in algorithm analysis (\\(O(|V| + |E|)\\) for graphs, for example) and in university-level texts on combinatorics and set theory.\nWe say a set is finite when its cardinality is a nonnegative integer (\\(0, 1, 2, \\ldots\\)). Saying \u0026ldquo;natural number\u0026rdquo; would not be enough: in this series, as we saw in the first article, \\(0 \\notin \\mathbb{N}\\), and the empty set, whose cardinality is 0, would be left out. When the enumeration does not end, the set is infinite (like \\(\\mathbb{N}\\), \\(\\mathbb{Z}\\), \\(\\mathbb{R}\\)).\nExamples \\(A = \\{x \\in \\mathbb{Z} \\mid |x| \\leq 3\\} = \\{-3,-2,-1,0,1,2,3\\}\\) → \\(n(A) = 7\\) \\(B = \\{x \\in \\mathbb{N} \\mid x \\text{ is even}\\} = \\{2,4,6,\\ldots\\}\\) → infinite \\(\\emptyset\\) → \\(n(\\emptyset) = 0\\) And what about infinite sets: do they all have the same size? The question seems meaningless, but answering it is what led Georg Cantor to create set theory.\nCardinality of infinite sets To compare sets without counting them, Cantor used bijections: two sets have the same cardinality when it is possible to pair their elements one to one, with no one left over on either side. For finite sets, this is the same as having the same number of elements; for infinite ones, it is the definition that works.\n\\(\\mathbb{Z}\\) has the same size as \\(\\mathbb{N}\\), even though it seems to have \u0026ldquo;twice as many\u0026rdquo; elements. Just list the integers alternating signs: \\(1 \\mapsto 0\\), \\(2 \\mapsto 1\\), \\(3 \\mapsto -1\\), \\(4 \\mapsto 2\\), \\(5 \\mapsto -2\\), and so on. Every integer appears exactly once in the list. \\(\\mathbb{Q}\\) also has the same size as \\(\\mathbb{N}\\): the fractions \\(p/q\\) can be laid out in an infinite table and traversed diagonally, skipping the repeated ones (\\(2/4\\) has already appeared as \\(1/2\\)). \\(\\mathbb{R}\\) is larger. By the diagonal argument (Cantor, 1891, originally formulated for infinite binary sequences and adapted here to decimals), any list of real numbers leaves some number out: just build a number whose first decimal digit differs from that of the first number in the list, the second from that of the second, and so on, using only digits other than 0 and 9 to avoid repeating decimals such as \\(0.4999\\ldots = 0.5\\). Infinite sets with the cardinality of \\(\\mathbb{N}\\) are called countable, and this cardinality has a name: \\(\\aleph_0\\) (aleph-null). Thus, \\(n(\\mathbb{N}) = n(\\mathbb{Z}) = n(\\mathbb{Q}) = \\aleph_0\\), while \\(n(\\mathbb{R}) \u0026gt; \\aleph_0\\). More at Cardinality, Countable set, and Aleph number.\nCantor and the resistance to infinity These ideas of Georg Cantor (1845 to 1918) were received with hostility by some of the mathematicians of the time. Leopold Kronecker, who only accepted objects built in a finite number of steps from the natural numbers, went as far as calling him a \u0026ldquo;scientific charlatan\u0026rdquo; and a \u0026ldquo;corrupter of youth.\u0026rdquo; Cantor had his first episode of depression in 1884, went through several hospitalizations, and died in a sanatorium in 1918. Recognition came during his lifetime and grew afterward: in 1926, David Hilbert wrote that \u0026ldquo;no one shall expel us from the paradise that Cantor has created.\u0026rdquo;\nIt is worth noting that being finite does not mean being easy to count. The set of all people born before the year 2000 is finite, but its exact cardinality is not immediately available. What we want are formulas that compute cardinalities from simpler data.\nSum Rule: Disjoint Sets # Two sets are disjoint when they have no element in common, that is, when \\(A \\cap B = \\emptyset\\). This concept was introduced in the article Venn Diagrams and Operations; the figure below recaps it visually.\nDisjoint sets: A ∩ B = ∅ The simplest counting case occurs when the sets are disjoint: with no overlap, we just add.\nSum Rule If \\(A \\cap B = \\emptyset\\), then: $$n(A \\cup B) = n(A) + n(B)$$Generalizing: if \\(A_1, A_2, \\ldots, A_k\\) are pairwise disjoint, then: $$n(A_1 \\cup A_2 \\cup \\cdots \\cup A_k) = n(A_1) + n(A_2) + \\cdots + n(A_k)$$The disjointness condition is essential. When there is an intersection, we need a more careful formula.\nInclusion-Exclusion Principle # When the sets are not disjoint, the plain sum \\(n(A) + n(B)\\) counts the elements of the intersection twice. The Inclusion-Exclusion Principle (PIE; see the Wikipedia entry) fixes this systematically: it adds the individual sizes, subtracts the pairwise overlaps, adds back the triple ones, alternating signs until each element is counted exactly once.\nTwo sets # Formula for two sets $$n(A \\cup B) = n(A) + n(B) - n(A \\cap B)$$The Venn diagram below helps visualize why the intersection is counted twice:\nThe intersection region A ∩ B is counted twice when adding n(A) + n(B) Why subtract the intersection? When adding \\(n(A)\\) and \\(n(B)\\), each element that belongs to both sets is counted twice, once when we count \\(A\\) and once when we count \\(B\\). Subtracting \\(n(A \\cap B)\\) restores the correct count.\nTo derive the formula, we decompose the union into three disjoint regions:\n$$A \\cup B = (A - B) \\cup (A \\cap B) \\cup (B - A)$$The three colored regions (A−B, A∩B, and B−A) are disjoint and cover the whole union By the sum rule:\n$$n(A \\cup B) = n(A - B) + n(A \\cap B) + n(B - A)$$But also:\n$$n(A - B) = n(A) - n(A \\cap B) \\qquad \\text{and} \\qquad n(B - A) = n(B) - n(A \\cap B)$$Substituting:\n$$n(A \\cup B) = \\bigl[n(A) - n(A \\cap B)\\bigr] + n(A \\cap B) + \\bigl[n(B) - n(A \\cap B)\\bigr] = n(A) + n(B) - n(A \\cap B)$$ Application in analytics: unique users (MAU) Product teams track the MAU (Monthly Active Users): the number of unique users who accessed the platform in a given month. Suppose a streaming service records:\n\\(n(Web) = 50{,}000\\) visits through the browser \\(n(App) = 40{,}000\\) visits through the mobile app Adding the two values directly would give 90,000, but many users access the service through both channels. If \\(n(Web \\cap App) = 15{,}000\\), the real number of unique users is:\n$$n(Web \\cup App) = 50{,}000 + 40{,}000 - 15{,}000 = \\mathbf{75{,}000}$$The Inclusion-Exclusion Principle corrects the double counting that inflates the metric, the same role it plays in theory.\nApplication in data engineering: deduplication in ETL pipelines (Extract, Transform, Load) When integrating data from multiple sources, such as a CRM (Customer Relationship Management), an e-commerce platform, and a support system, a company needs to know how many unique customers it has in total. Each source keeps its own list and the same customers may appear in more than one system.\nIf \\(n(\\text{CRM}) = 12{,}000\\), \\(n(\\text{Store}) = 8{,}000\\), and \\(n(\\text{CRM} \\cap \\text{Store}) = 3{,}000\\), the real number of distinct customers is:\n$$n(\\text{CRM} \\cup \\text{Store}) = 12{,}000 + 8{,}000 - 3{,}000 = \\mathbf{17{,}000}$$In ETL pipelines, the PIE makes it possible to get the total of unique records directly from partial counts, without having to build or scan the full unified list.\nThree sets # Formula for three sets $$n(A \\cup B \\cup C) = n(A) + n(B) + n(C) - n(A \\cap B) - n(A \\cap C) - n(B \\cap C) + n(A \\cap B \\cap C)$$The logic is the same: we add the individual sizes, subtract the pairwise overlaps (which were counted twice), and add back the triple overlap (which was subtracted once more than it should have been). The following figure identifies each of these regions in the Venn diagram:\nIn gold: the three pairwise intersections that are subtracted (A∩B, A∩C, B∩C). In red: the triple intersection A∩B∩C, added back to compensate for the double count A direct derivation: apply the two-set case to \\((A \\cup B) \\cup C\\), using \\(n(A \\cup B)\\) already worked out and the identity \\((A \\cup B) \\cap C = (A \\cap C) \\cup (B \\cap C)\\).\nTo see why the formula is right, it is worth looking at the eight regions into which three sets divide the universe, numbered as in the article on Venn diagrams:\nThe 2³ = 8 regions of three sets: each one is a combination of membership in A, B, and C, from region 1 (only A) to region 8 (outside all three) An element of region 7, which is in all three sets, is added 3 times in \\(n(A) + n(B) + n(C)\\), subtracted 3 times in the pairwise intersections, and added back 1 time in the triple one: \\(3 - 3 + 1 = 1\\). An element of regions 4, 5, or 6, which is in two sets, is added 2 times, subtracted 1 time, and does not appear in the triple one: \\(2 - 1 = 1\\). An element of regions 1, 2, or 3 is added once and appears in no other term, and those of region 8 enter no term at all. Each element of the union is counted exactly once.\nApplication in QA (Quality Assurance): unique bugs detected by three test suites A quality team maintains three independent suites: unit tests (\\(U\\)), integration tests (\\(I\\)), and end-to-end tests (\\(E\\)). They want to know how many distinct bugs were found in total, without counting twice the bugs detected by more than one suite.\nAfter a round of tests:\n\\(n(U) = 45\\), \\(n(I) = 38\\), \\(n(E) = 27\\) \\(n(U \\cap I) = 12\\), \\(n(U \\cap E) = 8\\), \\(n(I \\cap E) = 10\\) \\(n(U \\cap I \\cap E) = 3\\) $$n(U \\cup I \\cup E) = 45 + 38 + 27 - 12 - 8 - 10 + 3 = \\mathbf{83}$$Adding the individual totals would give 110, which is 27 more than the real number, exactly the number of bugs counted in duplicate (or triplicate) without the PIE correction.\nFour sets # The formula for four sets follows the same pattern, alternating signs:\n$$ \\begin{aligned} n(A_1 \\cup A_2 \\cup A_3 \\cup A_4) ={}\u0026 n(A_1)+n(A_2)+n(A_3)+n(A_4) \\\\ \u0026- n(A_1 \\cap A_2)-n(A_1 \\cap A_3)-n(A_1 \\cap A_4) \\\\ \u0026- n(A_2 \\cap A_3)-n(A_2 \\cap A_4)-n(A_3 \\cap A_4) \\\\ \u0026+ n(A_1 \\cap A_2 \\cap A_3)+n(A_1 \\cap A_2 \\cap A_4) \\\\ \u0026+ n(A_1 \\cap A_3 \\cap A_4)+n(A_2 \\cap A_3 \\cap A_4) \\\\ \u0026- n(A_1 \\cap A_2 \\cap A_3 \\cap A_4) \\end{aligned} $$ General formula # For \\(k\\) sets, the general formula of the Inclusion-Exclusion Principle is:\n$$n\\left(\\bigcup_{i=1}^{k} A_i\\right) = \\sum_{i} n(A_i) - \\sum_{i \u003c j} n(A_i \\cap A_j) + \\sum_{i \u003c j \u003c l} n(A_i \\cap A_j \\cap A_l) - \\cdots + (-1)^{k+1} n(A_1 \\cap \\cdots \\cap A_k)$$The pattern is clear: we add the individual sizes, subtract the pairwise intersections, add the triple ones, and so on, alternating signs.\nClassic example: derangements In a gift exchange, \\(n\\) people put their gifts in a box and each one draws one. In how many draws does nobody get their own gift? A permutation like this, with no element in its original position, is called a derangement, and the number of them is denoted \\(D_n\\).\nThe total number of draws is \\(n!\\), the number of permutations of the \\(n\\) gifts. Let \\(A_i\\) be the set of draws in which person \\(i\\) gets their own gift; we want the draws outside \\(A_1 \\cup \\cdots \\cup A_n\\).\n\\(n(A_i) = (n-1)!\\): the gift of \\(i\\) is fixed and the other \\(n-1\\) are permuted. The intersection of \\(k\\) of these sets fixes \\(k\\) gifts and has \\((n-k)!\\) elements. There are \\(\\binom{n}{k}\\) ways to choose the \\(k\\) people, so the terms with \\(k\\) sets add up to \\(\\binom{n}{k}(n-k)! = \\frac{n!}{k!}\\). By the PIE, with the sign \\((-1)^{k+1}\\) at each level, and taking the union out of the total:\n$$D_n = n! - \\sum_{k=1}^{n} (-1)^{k+1} \\frac{n!}{k!} = n! \\sum_{k=0}^{n} \\frac{(-1)^k}{k!}$$For \\(n = 4\\): \\(D_4 = 24 - 24 + 12 - 4 + 1 = 9\\). You can check by listing the 24 permutations of \\(1234\\): only nine leave no digit in its place.\nThe sum \\(\\sum_{k} (-1)^k/k!\\) converges to \\(1/e\\). The fraction of draws in which nobody gets their own gift tends to \\(1/e\\), about 36.8%. Rounded to one decimal place, it already gives 36.8% from 6 people on: it makes no difference whether the group has 10 or 10 thousand. More at Derangement.\nApplication in data science: Jaccard similarity A widely used metric for comparing sets (in recommender systems, plagiarism detection, and document clustering) is the Jaccard similarity:\n$$J(A, B) = \\frac{n(A \\cap B)}{n(A \\cup B)}$$The denominator is rarely computed by materializing the union set. The Inclusion-Exclusion Principle lets us get it directly from counts:\n$$n(A \\cup B) = n(A) + n(B) - n(A \\cap B)$$Thus, the similarity can be computed from three integers, without building the set \\(A \\cup B\\) explicitly.\nExample: two users of a streaming platform. User X watched the movies \\(\\{\\text{A, B, C, D}\\}\\) and user Y watched \\(\\{\\text{C, D, E, F}\\}\\). The movies in common are \\(\\{\\text{C, D}\\}\\), so \\(n(X \\cap Y) = 2\\). By the PIE:\n$$n(X \\cup Y) = 4 + 4 - 2 = 6 \\implies J(X, Y) = \\frac{2}{6} \\approx 0.33$$The closer to 1, the more similar the tastes, without having to list all the movies watched by at least one of the two.\nAt large scale, techniques such as MinHash estimate \\(J(A, B)\\) directly, bypassing the PIE, taking advantage of the fact that the probability that two sets share the same minimum hash equals the Jaccard similarity itself.\nApplication in databases: cardinality estimation in query planners SQL query optimizers (here, PostgreSQL\u0026rsquo;s) need to estimate, before executing, how many rows each operation will return: this estimate determines the order of the JOINs and which index to use. When a WHERE clause combines two predicates with OR, the optimizer needs to estimate \\(n(A \\cup B)\\), where \\(A\\) and \\(B\\) are the sets of rows that satisfy each predicate individually.\nWithout the Inclusion-Exclusion Principle, the optimizer would overestimate the result by adding the selectivities. With it, it uses:\n$$n(A \\cup B) \\approx n(A) + n(B) - n(A \\cap B)$$The estimates of \\(n(A)\\) and \\(n(B)\\) come from the column histograms and the most-common-values lists kept by the database. For \\(n(A \\cap B)\\), the default behavior is to assume independence between the predicates: if 30% of the rows satisfy predicate \\(A\\) and 20% satisfy \\(B\\), the optimizer assumes the two criteria are unrelated and estimates that 6% of the rows satisfy both, as if they were independent events (studied in probability). Extended statistics (such as CREATE STATISTICS in PostgreSQL) refine this estimate when the attributes are correlated, but require explicit configuration.\nThe PIE therefore appears in the estimation of any query with OR predicates in PostgreSQL, which computes \\(s_1 + s_2 - s_1 s_2\\) for the selectivities \\(s_1\\) and \\(s_2\\), invisible to the developer but present in the optimizer\u0026rsquo;s arithmetic.\nSummary Table # The formulas developed throughout the article, gathered for quick reference:\nSituation Formula Disjoint (\\(A \\cap B = \\emptyset\\)) \\(n(A \\cup B) = n(A) + n(B)\\) Two sets \\(n(A \\cup B) = n(A) + n(B) - n(A \\cap B)\\) Three sets \\(n(A \\cup B \\cup C) = \\sum n - \\sum n(\\cap_2) + n(\\cap_3)\\) Complement \\(n(U - A) = n(U) - n(A)\\) Difference \\(n(A - B) = n(A) - n(A \\cap B)\\) In the three-set row, \\(\\sum n\\) abbreviates the sum of the three individual sizes, \\(\\sum n(\\cap_2)\\) is the sum of the three pairwise intersections (\\(n(A \\cap B) + n(A \\cap C) + n(B \\cap C)\\)), and \\(n(\\cap_3)\\) is the triple intersection \\(n(A \\cap B \\cap C)\\). The complement and difference rows follow directly from the sum rule: just decompose the larger set into two disjoint parts and isolate the desired term.\nExercises # Exercise 1: Divisibility in \\(\\{1, \\ldots, 100\\}\\) How many numbers from 1 to 100 are divisible by neither 2 nor 5?\nSolution Let \\(C = \\{1, \\ldots, 100\\}\\), \\(A\\) = divisible by 2, \\(B\\) = divisible by 5.\n\\(n(A) = 50\\) (every second number) \\(n(B) = 20\\) (every fifth number) \\(n(A \\cap B) = 10\\) (divisible by 10) $$n(A \\cup B) = 50 + 20 - 10 = 60$$The ones divisible by neither 2 nor 5:\n$$n(C) - n(A \\cup B) = 100 - 60 = \\mathbf{40}$$ Exercise 2: Proof Let \\(A\\) and \\(B\\) be two subsets of \\(U\\) with \\(B \\subseteq A\\). Prove, using the sum rule, that \\(n(A - B) = n(A) - n(B)\\).\nProof Since \\(B \\subseteq A\\), we have \\(A \\cup B = A\\). We can rewrite \\(A\\) as the disjoint union \\(A = (A - B) \\cup B\\). By the sum rule:\n$$n(A) = n(A - B) + n(B)$$Therefore \\(n(A - B) = n(A) - n(B)\\). \\(\\blacksquare\\)\nExercise 3: Multiples of 3 or 7 up to 100 How many integers from 1 to 100 are divisible by 3 or by 7?\nSolution Let \\(A\\) = multiples of 3, \\(B\\) = multiples of 7, in \\(\\{1,\\ldots,100\\}\\).\n\\(n(A) = \\lfloor 100/3 \\rfloor = 33\\) \\(n(B) = \\lfloor 100/7 \\rfloor = 14\\) \\(n(A \\cap B)\\) = multiples of \\(\\text{lcm}(3,7) = 21\\): \\(\\{21, 42, 63, 84\\}\\), so \\(n(A \\cap B) = 4\\) $$n(A \\cup B) = 33 + 14 - 4 = \\mathbf{43}$$ Exercise 4: Multiples in \\((1, 60)\\) Consider the integers from 2 to 59. Determine how many are:\n(i) divisible by 2 and by 3 (ii) divisible by 2 or by 3 (iii) divisible by neither 2 nor 3\n(iv) odd and divisible by 3, or divisible by 2 (v) divisible by 2, 3, or 5\nSolutions Let \\(A\\) = even numbers, \\(B\\) = multiples of 3, all in \\((1,60)\\) (excluding the endpoints):\n\\(n(A) = 29\\), \\(n(B) = 19\\), \\(n(A \\cap B) = 9\\) (multiples of 6).\n(i) \\(n(A \\cap B) = \\mathbf{9}\\)\n(ii) \\(n(A \\cup B) = 29 + 19 - 9 = \\mathbf{39}\\)\n(iii) Total = 58 numbers. Not in \\(A \\cup B\\): \\(58 - 39 = \\mathbf{19}\\)\n(iv) \\(D\\) = odd multiples of 3 = \\(\\{3,9,15,21,27,33,39,45,51,57\\}\\), \\(n(D) = 10\\). Since \\(A \\cap D = \\emptyset\\): \\(n(A \\cup D) = 29 + 10 = \\mathbf{39}\\)\n(v) \\(E\\) = multiples of 5 in \\((1,60)\\) = \\(\\{5,10,\\ldots,55\\}\\), \\(n(E) = 11\\). \\(n(A \\cap E) = 5\\), \\(n(B \\cap E) = 3\\), \\(n(A \\cap B \\cap E) = 1\\).\n$$n(A \\cup B \\cup E) = 29+19+11-9-5-3+1 = \\mathbf{43}$$ Exercise 5: Market research 200 people were asked about television prices: 40% asked about brand A, 35% about brand B, 10% about both, and 35% asked about neither brand. Determine:\n(i) how many asked about A or B\n(ii) how many asked about A and not about B\nSolutions In absolute values: \\(n(A) = 80\\), \\(n(B) = 70\\), \\(n(A \\cap B) = 20\\).\n(i) \\(n(A \\cup B) = 80 + 70 - 20 = \\mathbf{130}\\) people\n(ii) \\(n(A - B) = n(A) - n(A \\cap B) = 80 - 20 = \\mathbf{60}\\) people\nSieve of Eratosthenes # Exercises 1 and 3 and item (v) of 4 follow the same scheme: define a set by divisibility, compute its cardinality with \\(\\lfloor n/k \\rfloor\\), use the PIE to gather multiples of different primes without double counting and, when necessary, take the complement to count the elements that satisfy none of the conditions. This pattern, counting by the complement after applying the PIE, is recurrent in combinatorics.\nThese divisibility exercises apply exactly the same reasoning as the Sieve of Eratosthenes, the oldest known algorithm for finding all prime numbers up to a limit \\(n\\), proposed by the Greek mathematician Eratosthenes of Cyrene in the 3rd century BC.\nDefinition A prime number is a natural number that has exactly two distinct natural divisors: the number 1 and itself.\nThe first primes are 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, …\nHow it works # Given an integer \\(n \u0026gt; 1\\), Eratosthenes\u0026rsquo; method:\n1 Build the list Create a list of consecutive integers from 2 to \\(n\\). 2 Start with 2 Let \\(p = 2\\), the smallest prime. 3 Mark the multiples of \\(p\\) Mark as composite all multiples of \\(p\\) starting from \\(2p\\): \\(2p,\\ 3p,\\ 4p,\\ \\ldots\\). \\(p\\) itself is not marked. 4 Move on to the next prime Find the smallest unmarked number in the list greater than \\(p\\). If there is none, stop. Otherwise, \\(p\\) takes this new value and go back to step 3. 5 Read off the primes When finished, the unmarked numbers are all the primes up to \\(n\\). The central idea: every value taken by \\(p\\) is necessarily prime, because if it were composite, it would already have been marked as a multiple of some smaller prime. Note that, in this basic algorithm, some numbers may be marked more than once: 15, for example, is marked both when processing \\(p = 3\\) and when processing \\(p = 5\\).\nOptimization: in step 3, it is enough to start at \\(p^2\\) instead of \\(2p\\), since every multiple \\(k \\cdot p\\) with \\(k \u0026lt; p\\) has a prime factor \\(q \\leq k \u0026lt; p\\) by which it was already marked in an earlier stage. With that, it is also enough to run step 3 while \\(p^2 \\leq n\\): upon reaching a \\(p\\) with \\(p^2 \u0026gt; n\\), all the numbers still unmarked are necessarily prime.\nFor example, when processing \\(p = 5\\), the multiples before \\(p^2 = 25\\) have already been marked: \\(10 = 2 \\cdot 5\\) was marked when \\(p = 2\\); \\(15 = 3 \\cdot 5\\), when \\(p = 3\\); \\(20 = 4 \\cdot 5 = 2^2 \\cdot 5\\), again when \\(p = 2\\). The first multiple of 5 that can still be unmarked is \\(25 = 5^2\\), hence the start at \\(p^2\\).\nThe animation below illustrates the complete algorithm with the optimization applied, for all primes below 121:\nSieve of Eratosthenes: steps of the algorithm for primes below 121, including the optimization of starting the marking from the square of the prime. Source: Wikimedia Commons Example for \\(n = 30\\), with primes in bold and composites struck through:\n2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 With \\(p = 2\\): 4, 6, 8, …, 30 are marked. With \\(p = 3\\): 9, 15, 21, 27 are marked (the even numbers were already marked). With \\(p = 5\\): 25 is marked. The next prime would be \\(p = 7\\), but \\(7^2 = 49 \u0026gt; 30\\): the only multiples of 7 in the range (14, 21, and 28) are already marked by smaller primes (2 and 3). All the numbers still unmarked are prime. Ten remain: the primes from 2 to 29.\nAlgorithm # The pseudocode below directly incorporates the \\(p^2\\) optimization:\nInput: integer n \u0026gt; 1 Output: all primes from 2 to n A ← array[2..n] initialized with True For i from 2 to √n: If A[i] is True: For j from i² to n with step i: A[j] ← False Return all i with A[i] = True The flowchart below represents the same algorithm visually:\n\u0026#x2197; flowchart TD A([Start]) --\u003e B[\"A ← array[2..n] = True\ni ← 2\"] B --\u003e C{\"i ≤ √n ?\"} C --\u003e|No| G[/\"Return all i\nwith A[i] = True\"/] G --\u003e H([End]) C --\u003e|Yes| D{\"A[i] = True ?\"} D --\u003e|No| F[\"i ← i + 1\"] F --\u003e C D --\u003e|Yes| E[\"j ← i²\"] E --\u003e I{\"j ≤ n ?\"} I --\u003e|No| F I --\u003e|Yes| J[\"A[j] ← False\"] J --\u003e K[\"j ← j + i\"] K --\u003e I In the pseudocode, the same optimization appears from two angles: the inner loop starts at \\(i^2\\) because earlier multiples have already been marked; the outer loop stops at \\(\\sqrt{n}\\) because, when \\(i \u0026gt; \\sqrt{n}\\), we have \\(i^2 \u0026gt; n\\), and there is no multiple of \\(i\\) from \\(i^2\\) on within the range. The two conditions are equivalent.\nTime complexity: \\(O(n \\log \\log n)\\). The intuition: for each prime \\(p \\leq n\\), the inner loop performs about \\(n/p\\) steps. The total cost is proportional to \\(n \\sum_{p \\leq n} 1/p\\), and by Mertens' theorem this sum grows like \\(\\log \\log n\\), much more slowly than \\(\\log n\\) (the full harmonic series). In practice, for \\(n = 10^9\\), \\(\\ln \\ln n \\approx 3\\): the algorithm is almost linear.\nSpace complexity: the pseudocode above allocates an array of \\(n\\) positions, hence \\(O(n)\\), infeasible for \\(n = 10^9\\). The Segmented Sieve is a variant that reduces the space to \\(O(\\sqrt{n})\\) without increasing the time complexity. The idea in two phases: first, run the classic sieve up to \\(\\sqrt{n}\\) to obtain all the small primes (this array fits in \\(O(\\sqrt{n})\\)); then, divide \\([2, n]\\) into blocks of size \\(\\approx \\sqrt{n}\\) (sized to fit in the processor\u0026rsquo;s cache memory) and process one block at a time, marking the multiples of the small primes that fall in that range. The resulting pseudocode is longer than the one above, but the principle is the same.\nAdditions only: the sieve advances by counting in increments of \\(p\\), with no divisions or divisibility tests. This is what distinguishes it from trial division, which tests each candidate individually, and is the basis of its efficiency.\nThe connection with the PIE # Why does the sieve work mathematically? Every composite number \\(m \\leq n\\) has at least one prime factor \\(p \\leq \\sqrt{m} \\leq \\sqrt{n}\\), so every composite falls in some \\(A_p\\) and is eliminated. What remains are the primes.\nBack to the example with \\(n = 30\\): \\(21 = 3 \\times 7\\) is composite and \\(\\sqrt{21} \\approx 4.6\\), so it has a prime factor \\(\\leq 4.6\\) (indeed, 3) and was eliminated when \\(p = 3\\). Meanwhile \\(25 = 5^2\\) has \\(\\sqrt{25} = 5\\), with prime factor \\(5 \\leq 5\\), eliminated when \\(p = 5\\). No composite escapes: there is always a prime factor small enough to have marked it before the outer loop ends.\nEliminating the multiples of a prime \\(p\\) amounts to defining:\n$$A_p = \\{k \\in \\{1, \\ldots, n\\} : k \\text{ is a multiple of } p\\}$$In words: \\(A_p\\) is the set of all integers from 1 to \\(n\\) that the sieve would mark when processing the prime \\(p\\). For example, for \\(p = 3\\) and \\(n = 30\\), we have \\(A_3 = \\{3, 6, 9, 12, \\ldots, 30\\}\\). Each step of the algorithm does nothing more than build this set and strike out its elements.\nThe survivors of the sieve are the integers from 1 to \\(n\\) that belong to no \\(A_p\\), that is, the complement of the union of all these sets. Calling \\(p_1, \\ldots, p_r\\) the primes up to \\(\\sqrt{n}\\), the count of the survivors is:\n$$n - n(A_{p_1} \\cup \\cdots \\cup A_{p_r})$$(The number 1 also survives, since it is not divisible by any prime; \\(p_1, \\ldots, p_r\\) themselves are excluded, since each \\(p_i\\) belongs to its own \\(A_{p_i}\\). The sieve starts at 2, which takes care of both in practice.)\nTo compute \\(n(A_{p_1} \\cup \\cdots \\cup A_{p_r})\\), we apply the PIE. This requires knowing the cardinalities of the individual sets and of the intersections. The key is that the intersection of two sets of multiples is itself a set of multiples: \\(A_p \\cap A_q\\) contains exactly the integers divisible by \\(p\\) and by \\(q\\), that is, the multiples of the product \\(pq\\) (which is the lcm of two distinct primes). Therefore:\nThe cardinality of \\(A_p\\) is the number of multiples of \\(p\\) up to \\(n\\), given by the integer part \\(\\lfloor n/p \\rfloor\\). The cardinality of the intersection \\(A_p \\cap A_q\\) is the number of multiples of \\(pq\\) up to \\(n\\), given by \\(\\lfloor n/(pq) \\rfloor\\). $$n(A_p) = \\left\\lfloor \\frac{n}{p} \\right\\rfloor \\qquad n(A_p \\cap A_q) = \\left\\lfloor \\frac{n}{pq} \\right\\rfloor$$You computed exactly this in item (v) of Exercise 4: with \\(A\\) = even numbers, \\(B\\) = multiples of 3, and \\(E\\) = multiples of 5 in the range \\((1, 60)\\), you applied the PIE to \\(A_2 \\cup A_3 \\cup A_5\\) and got 43. The exercise stopped there, but the sieve takes one more step: it takes the complement, \\(58 - 43 = 15\\), to count the survivors (the numbers from 2 to 59 not divisible by 2, 3, or 5, which include all primes greater than 5 in that range). The Sieve of Eratosthenes is this reasoning generalized to any \\(n\\) and any set of primes.\nThe Sieve of Eratosthenes is an algorithm: it iteratively marks multiples and counts what remains. What we did in this section was different: we expressed the same result as a closed-form formula via the PIE, directly computing \\(n - n(A_{p_1} \\cup \\cdots \\cup A_{p_r})\\) without simulating the process step by step. This mathematical formalization is historically known as the Legendre sieve, named in honor of Adrien-Marie Legendre, who in the 19th century made explicit the PIE structure behind the sieve. The central formula of the method is called Legendre\u0026rsquo;s identity, and it generalizes the expression above using the Möbius function \\(\\mu(d)\\). This function encodes exactly the alternating sign pattern of the PIE: for a number \\(d\\) that is a product of \\(k\\) distinct primes, \\(\\mu(d) = (-1)^k\\); if \\(d\\) contains some squared factor, \\(\\mu(d) = 0\\). With this, the whole PIE expansion (add the individual ones, subtract the pairs, add the triples, and so on) condenses into a single sum, which already gives the count of the survivors:\n$$n - n(A_{p_1} \\cup \\cdots \\cup A_{p_r}) = \\sum_{d \\mid P} \\mu(d) \\left\\lfloor \\frac{n}{d} \\right\\rfloor$$where \\(P = p_1 \\cdot p_2 \\cdots p_r\\) is the product of the primes up to \\(\\sqrt{n}\\) and the sum runs over all divisors \\(d\\) of \\(P\\), including \\(d = 1\\). It is this term that plays the role of the initial \\(n\\): \\(\\mu(1) \\lfloor n/1 \\rfloor = n\\). The prime divisors (\\(\\mu = -1\\)) subtract the individual sets, the products of two primes (\\(\\mu = +1\\)) add back the pairwise intersections, and so on. It is the same arithmetic as the PIE, rewritten compactly.\nFor \\(n = 30\\), the primes up to \\(\\sqrt{30}\\) are 2, 3, and 5, and \\(P = 30\\). The eight divisors of 30 give:\n$$\\underbrace{30}_{d=1} - \\underbrace{15 - 10 - 6}_{d \\in \\{2, 3, 5\\}} + \\underbrace{5 + 3 + 2}_{d \\in \\{6, 10, 15\\}} - \\underbrace{1}_{d=30} = 8$$These are exactly the eight survivors \\(\\{1, 7, 11, 13, 17, 19, 23, 29\\}\\): 1 is in, and the primes 2, 3, and 5 are out, as explained above.\nThe figure below puts this computation on the numbers themselves. Each integer from 1 to 30 gets a mark for each set \\(A_2\\), \\(A_3\\), \\(A_5\\) it belongs to, and those left with no mark are the survivors. Counting the marks of each color gives \\(15\\), \\(10\\), and \\(6\\). The pairwise intersections add up to \\(5 + 3 + 2 = 10\\): they are the seven cells with two marks (6, 10, 12, 15, 18, 20, and 24) plus 30, which has three marks and is therefore in all three pairwise intersections, and is, on its own, the triple one.\nFrom 1 to 30, each number gets one mark for each set A₂, A₃, and A₅ it belongs to; the eight with no mark are the survivors, and the PIE computation appears below Next steps # With cardinality and the Inclusion-Exclusion Principle, we close the theoretical block of this stage of the series: defining, representing, operating on, and counting sets without double counting.\nIn the next article of the series, the same language appears in code: we will see how membership, inclusion, union, intersection, and cardinality translate directly into Python using set and frozenset, connecting theory to practice.\nSee you there!\n","date":"2026-06-11","externalUrl":null,"permalink":"/2026/06/cardinality/","section":"Blog","summary":"Adding the sizes of two sets counts the intersection twice. The Inclusion-Exclusion Principle fixes the count, for two, three, or any number of sets. We will also see infinities of different sizes, derangements, and the Sieve of Eratosthenes.","title":"Cardinality: Counting the Elements of a Set","type":"posts"},{"content":" In the previous article we saw what a set is and how to define its elements precisely. Now we take one step further: we will learn to operate on sets (combine them, find what they have in common, subtract one from another) and to visualize them with Venn diagrams.\nThese operations show up everywhere: in queries written in SQL (Structured Query Language), in data filters, and in programming logic. For example, a predicate such as WHERE a IN (SELECT ...) AND b NOT IN (...) already combines the inclusion and exclusion of sets.\nPractical reading: databases and search In relational databases, UNION, INTERSECT, and EXCEPT correspond, respectively, to union, intersection, and difference (see the PostgreSQL documentation on these operators). In search engines, OR widens the result set, AND narrows it, and exclusion with - removes the unwanted terms.\nThe Universal Set and Venn Diagrams # Universal Set The universal set \\(U\\) is the set that contains all the objects relevant in the context of a problem. Every set in the problem is a subset of \\(U\\).\nWhen we write \\(A = \\{x \\in \\mathbb{N} \\mid x \\leq 50\\}\\), we are saying that \\(U = \\mathbb{N}\\) and we select within \\(U\\) only the elements that satisfy the property. Stating the universe explicitly avoids ambiguity.\nVenn diagrams are the geometric representation of this idea:\nThe universe \\(U\\) is represented by the outer rectangle Each subset is represented by a circular region inside the rectangle The relative position of the circles indicates the relationships between the sets The figure below shows the most elementary case: \\(A \\subset U\\). The circle of \\(A\\) lies entirely inside the rectangle \\(U\\): every element of \\(A\\) belongs to \\(U\\), but \\(U\\) contains elements outside \\(A\\).\nVenn diagram illustrating A ⊂ U: the circle of set A is contained in the rectangle of the universe U The diagrams are named in honor of the English mathematician John Venn (1834 to 1923), who introduced them in 1880.\nVenn diagram or Euler diagram? More than a century before Venn, Leonhard Euler already used circles to represent relationships between classes, in the Letters to a German Princess, written between 1760 and 1762 and published in 1768. The two drawings follow different rules:\nAn Euler diagram shows only the regions that can have elements. If \\(A \\subset U\\), the circle of \\(A\\) lies inside the rectangle; if two sets are disjoint, their circles do not even touch. A Venn diagram, in the strict sense, draws all the combinations of membership (\\(2^2 = 4\\) regions for two sets, \\(2^3 = 8\\) for three), including the empty ones. What is known about each region is indicated by shading or marking it. In current usage, even in mathematics books, \u0026ldquo;Venn diagram\u0026rdquo; has become the name for any drawing of this kind, and this article uses the term in that broad sense. Strictly speaking, the figure above and the figures for inclusion and disjoint sets throughout the text are Euler diagrams.\nThe figure below is a Venn diagram in the strict sense, for three sets. The eight regions always appear, even if some are empty, and each one corresponds to a combination of membership in \\(A\\), \\(B\\), and \\(C\\); the color indicates how many of the three sets the region is in. This numbering returns in Exercise 3 and in the next article, when we count the elements of a union of three sets.\nThe 2³ = 8 regions of three sets: each is a combination of membership in A, B, and C, from region 1 (only A) to region 8 (outside all three) Example: inclusion in a diagram Consider \\(U = \\mathbb{N}\\), \\(A = \\{x \\in \\mathbb{N} \\mid 10 \\leq x \\leq 100\\}\\), and \\(B = \\{x \\in \\mathbb{N} \\mid 15 \\leq x \\leq 50\\}\\).\nSince every element of \\(B\\) also belongs to \\(A\\), we have \\(B \\subset A \\subset U\\). In a Venn diagram, the circle of \\(B\\) lies entirely inside the circle of \\(A\\), which in turn is inside the rectangle \\(U\\):\nVenn diagram with three nested regions: B ⊂ A ⊂ U The Four Fundamental Operations # Union, intersection, difference, and complement are the basic operations of the algebra of sets (see the Wikipedia entry Set operations (Boolean)). For each one, we will see the definition, the diagram, examples, and the SQL correspondence.\nUnion # Definition The union of \\(A\\) and \\(B\\) gathers all the elements that belong to at least one of the two sets: $$A \\cup B = \\{x \\in U \\mid x \\in A\\ \\text{or}\\ x \\in B\\}$$In the Venn diagram, the union corresponds to the entire area covered by the two circles.\nShaded region representing A ∪ B: all the elements that belong to A, to B, or to both Example with elements If \\(A = \\{1,2,3,4,5,6\\}\\) and \\(B = \\{5,6,7,8,9\\}\\), then the elements \\(5\\) and \\(6\\) lie in the intersection \\(A \\cap B\\), while \\(1,2,3,4\\) lie only in \\(A\\) and \\(7,8,9\\) only in \\(B\\). The union joins all these regions, resulting in: $$A \\cup B = \\{1,2,3,4,5,6,7,8,9\\}$$Example with elements distributed in the regions only A, A ∩ B, and only B Example without intersection (disjoint sets) If \\(A = \\{1,2,3\\}\\) and \\(B = \\{7,8,9\\}\\), then \\(A \\cap B = \\emptyset\\): the circles are isolated in the universe \\(U\\), with no overlap. Even so, the union is made up of all the elements that are in \\(A\\) or in \\(B\\): $$A \\cup B = \\{1,2,3,7,8,9\\}$$Example of a union with disjoint A and B: separate circles in the rectangle of the universe Properties:\nProperty Statement Idempotence \\(A \\cup A = A\\) Identity element \\(A \\cup \\emptyset = A\\) Absorption by the universe \\(A \\cup U = U\\) Commutativity \\(A \\cup B = B \\cup A\\) Associativity \\((A \\cup B) \\cup C = A \\cup (B \\cup C)\\) Inclusion in the union \\(A \\subseteq A \\cup B\\) Equivalence (characteristic) \\(A \\subseteq B \\iff A \\cup B = B\\) Special case: if \\(W \\subseteq Z\\), then \\(Z \\cup W = Z\\).\nApplied example (minimum height) Consider, in a universe \\(U\\) of measured people: $$Z = \\{x \\in U \\mid \\text{height}(x) \\geq 1.75\\text{ m}\\}$$ $$W = \\{x \\in U \\mid \\text{height}(x) \\geq 1.90\\text{ m}\\}$$Every element of \\(W\\) also belongs to \\(Z\\), so \\(W \\subseteq Z\\). Therefore: $$Z \\cup W = Z$$Inclusion case: W is contained in Z, so the union Z ∪ W coincides with Z Application in SQL In SQL, the union corresponds to the UNION operator, which combines the results of two queries, eliminating duplicates. The query below returns all the customers from SP or RJ, that is, \\(A \\cup B\\):\nSELECT id FROM customers WHERE state = \u0026#39;SP\u0026#39; UNION SELECT id FROM customers WHERE state = \u0026#39;RJ\u0026#39; Intersection # Definition The intersection of \\(A\\) and \\(B\\) gathers only the elements that belong to both simultaneously: $$A \\cap B = \\{x \\in U \\mid x \\in A\\ \\text{and}\\ x \\in B\\}$$In the Venn diagram, the intersection is the overlapping area of the two circles.\nShaded region representing A ∩ B: only the elements common to A and B Example with elements If \\(A = \\{1,2,3,4,5,6\\}\\) and \\(B = \\{5,6,7,8,9\\}\\), only the elements common to both sets enter the intersection: $$A \\cap B = \\{5,6\\}$$Intersection with elements: only the region common to A and B is highlighted Example without intersection (disjoint sets) If \\(A = \\{1,2,3\\}\\) and \\(B = \\{7,8,9\\}\\), the sets share no elements, so: $$A \\cap B = \\emptyset$$Intersection of disjoint sets: there is no common region, so A ∩ B = ∅ Properties:\nProperty Statement Idempotence \\(A \\cap A = A\\) Intersection with the empty set \\(A \\cap \\emptyset = \\emptyset\\) Intersection with the universe \\(A \\cap U = A\\) Commutativity \\(A \\cap B = B \\cap A\\) Inclusion of the intersection \\(A \\cap B \\subseteq A\\) Equivalence (characteristic) \\(A \\subseteq B \\iff A \\cap B = A\\) Disjoint sets \\(A\\) and \\(B\\) are disjoint when \\(A \\cap B = \\emptyset\\): they share no element.\nSpecial case: if \\(W \\subseteq Z\\), then \\(Z \\cap W = W\\).\nApplied example (minimum height) Consider, in a universe \\(U\\) of measured people: $$Z = \\{x \\in U \\mid \\text{height}(x) \\geq 1.75\\text{ m}\\}$$ $$W = \\{x \\in U \\mid \\text{height}(x) \\geq 1.90\\text{ m}\\}$$Since \\(W \\subseteq Z\\), every element of \\(W\\) is already in \\(Z\\). Therefore: $$Z \\cap W = W$$Inclusion case: W is contained in Z, so the intersection Z ∩ W coincides with W Application in SQL The intersection corresponds to the INTERSECT operator. The query below returns only the customers who bought in both January and February, that is, \\(A \\cap B\\):\nSELECT customer_id FROM sales WHERE month = 1 INTERSECT SELECT customer_id FROM sales WHERE month = 2 Difference # Definition The difference \\(A - B\\) (read \u0026ldquo;\\(A\\) minus \\(B\\)\u0026rdquo;) contains the elements of \\(A\\) that are not in \\(B\\): $$A - B = \\{x \\in U \\mid x \\in A\\ \\text{and}\\ x \\notin B\\}$$In the Venn diagram, it is the part of \\(A\\) that does not overlap \\(B\\). Note that the operation is not commutative: \\(A - B \\neq B - A\\) in general.\nIn the literature, this operation also appears as the relative complement of \\(B\\) in \\(A\\).\nShaded region representing A - B: elements that are in A and not in B Example with elements (both directions) If \\(A = \\{1,2,3,4,5,6\\}\\) and \\(B = \\{5,6,7,8,9\\}\\): $$A - B = \\{1,2,3,4\\}$$ $$B - A = \\{7,8,9\\}$$Difference A - B with elements: only the elements exclusive to A remain Difference B - A with elements: only the elements exclusive to B remain Example without intersection (disjoint sets) If \\(A = \\{1,2,3\\}\\) and \\(B = \\{7,8,9\\}\\), then \\(A \\cap B = \\emptyset\\), and: $$A - B = \\{1,2,3\\} = A$$ $$B - A = \\{7,8,9\\} = B$$When the sets are disjoint, the subtrahend removes no element from the minuend, so the difference is always equal to the minuend itself (the left side of the operation): \\(A - B = A\\) and \\(B - A = B\\). In the diagrams, the highlighted region is the entire circle of the minuend:\nDisjoint sets: the region of A − B is the entire circle of A, so A − B = A Disjoint sets: the region of B − A is the entire circle of B, so B − A = B Properties:\nProperty Statement Difference with itself \\(A - A = \\emptyset\\) Removing the empty set \\(A - \\emptyset = A\\) Empty set as minuend \\(\\emptyset - A = \\emptyset\\) Equivalence (characteristic) \\(A \\subseteq B \\iff A - B = \\emptyset\\) Inclusion of the difference \\(A - B \\subseteq A\\) Useful relation: \\(A - B = A \\cap \\bar{B}\\) (we will see the complement next).\nIf \\(W \\subseteq Z\\), then \\(W - Z = \\emptyset\\).\nApplied example (minimum height, both directions) Consider, in a universe \\(U\\) of measured people: $$Z = \\{x \\in U \\mid \\text{height}(x) \\geq 1.75\\text{ m}\\}$$ $$W = \\{x \\in U \\mid \\text{height}(x) \\geq 1.90\\text{ m}\\}$$Since \\(W \\subseteq Z\\), we have: $$Z - W = \\{x \\in U \\mid 1.75 \\leq \\text{height}(x) \u003c 1.90\\}$$ $$W - Z = \\emptyset$$Difference Z - W: region of Z outside W Difference W - Z: empty because W is contained in Z Application in SQL The difference corresponds to the EXCEPT operator. A classic use is identifying records present in one table but absent from another, such as users who signed up but have not yet made any purchase (\\(A - B\\)):\nSELECT id FROM users EXCEPT SELECT DISTINCT user_id FROM ordersCombining the two directions of the difference (\\(A - B\\) and \\(B - A\\)) we obtain the symmetric difference, which retains only the elements exclusive to each set. We explore this operation in Exercise 2.\nComplement # Definition The complement of \\(A\\) with respect to \\(U\\) is everything that is in \\(U\\) but not in \\(A\\): $$\\bar{A} = U - A = \\{x \\in U \\mid x \\notin A\\}$$In the Venn diagram, the complement is the entire area of the rectangle that lies outside the circle of \\(A\\):\nShaded region representing the complement of A: all the elements of U that do not belong to A Properties:\nProperty Statement Involution \\(\\overline{\\bar{A}} = A\\) Complementarity \\(A \\cup \\bar{A} = U\\) Disjointness \\(A \\cap \\bar{A} = \\emptyset\\) Complement of the universe \\(\\bar{U} = \\emptyset\\) Complement of the empty set \\(\\bar{\\emptyset} = U\\) Examples of complement \\(U = \\mathbb{N}\\), \\(A = \\{x \\in \\mathbb{N} \\mid x \\leq 50\\}\\) → \\(\\bar{A} = \\{x \\in \\mathbb{N} \\mid x \u0026gt; 50\\}\\) \\(U = \\mathbb{Z}\\), \\(A = \\{x \\in \\mathbb{Z} \\mid x \u0026gt; 2\\}\\) → \\(\\bar{A} = \\{\\ldots, -2, -1, 0, 1, 2\\}\\) \\(U = \\mathbb{N}\\), \\(A = \\{x \\in \\mathbb{N} \\mid x \u0026gt; 2\\}\\) → \\(\\bar{A} = \\{1, 2\\}\\) Application in SQL The complement appears whenever a subset is excluded from the universe. The query below returns all the users without an active subscription, that is, the complement of the set of subscribers:\nSELECT id FROM users WHERE id NOT IN (SELECT user_id FROM subscriptions WHERE active = TRUE) If user_id can be NULL in subscriptions, the NOT IN returns no rows at all and stops being the complement. In that case, use NOT EXISTS.\nFundamental Identities # De Morgan\u0026rsquo;s Laws # De Morgan\u0026rsquo;s laws describe how the complement distributes over union and intersection. They are widely used in logic and programming, and if you have read the article on Boolean Algebra, you have already met them in the context of the logical operations AND, OR, and NOT. Here, the identities are expressed in terms of sets:\n$$\\overline{A \\cup B} = \\bar{A} \\cap \\bar{B}$$$$\\overline{A \\cap B} = \\bar{A} \\cup \\bar{B}$$In words: the complement of a union is the intersection of the complements; the complement of an intersection is the union of the complements. You use this when negating compound conditions: NOT (A OR B) is equivalent to NOT A AND NOT B.\nExample of logical refactoring Imagine a rule that grants access to whoever is an admin or has an active account and blocks everyone else. Written directly, the blocking condition is the negation of the access condition:\nif not (is_admin or is_active): block_access() By De Morgan\u0026rsquo;s First Law, the same condition can be rewritten as \u0026ldquo;is not an admin and the account is not active\u0026rdquo;:\nif not is_admin and not is_active: block_access() The two forms are logically equivalent. The first makes the access rule being negated visible; the second lists, one by one, the conditions that lead to blocking. Which one reads better depends on how the rule was conceived.\nThe figures below show each identity in steps, separately:\nStep-by-step visualization of De Morgan\u0026rsquo;s first law Step-by-step visualization of De Morgan\u0026rsquo;s second law Distributivity # $$A \\cap (B \\cup C) = (A \\cap B) \\cup (A \\cap C)$$$$A \\cup (B \\cap C) = (A \\cup B) \\cap (A \\cup C)$$Intersection distributes over union, and vice versa, exactly as multiplication distributes over addition in the numbers.\nThe following figures show, in steps, the two distributivity equalities:\nStep-by-step illustration of A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) Step-by-step illustration of A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) Proof of the second identity \\(A \\cup (B \\cap C) = (A \\cup B) \\cap (A \\cup C)\\):\nPart 1: Let us show that \\(A \\cup (B \\cap C) \\subseteq (A \\cup B) \\cap (A \\cup C)\\).\nLet \\(x \\in A \\cup (B \\cap C)\\). Then either \\(x \\in A\\), or \\(x \\in B \\cap C\\) (that is, \\(x \\in B\\) and \\(x \\in C\\)). In both cases, \\(x \\in A \\cup B\\) and \\(x \\in A \\cup C\\) simultaneously. Hence \\(x \\in (A \\cup B) \\cap (A \\cup C)\\). \\(\\square\\)\nPart 2: Let us show the opposite inclusion.\nLet \\(x \\in (A \\cup B) \\cap (A \\cup C)\\). Then \\(x \\in A \\cup B\\) and \\(x \\in A \\cup C\\). If \\(x \\in A\\), we are done. Otherwise, to belong to \\(A \\cup B\\) without being in \\(A\\), we must have \\(x \\in B\\); and to belong to \\(A \\cup C\\) without being in \\(A\\), we must have \\(x \\in C\\). Therefore \\(x \\in B \\cap C\\), and so \\(x \\in A \\cup (B \\cap C)\\). \\(\\square\\)\nThe first identity (\\(A \\cap (B \\cup C) = (A \\cap B) \\cup (A \\cap C)\\)) follows by an analogous argument, swapping the roles of \\(\\cup\\) and \\(\\cap\\). Try proving it as an exercise.\nAbsorption # $$A \\cup (A \\cap B) = A \\qquad \\text{and} \\qquad A \\cap (A \\cup B) = A$$ Other Useful Identities # $$A - B = A \\cap \\bar{B}$$Proof: \\(x \\in A - B\\) if and only if \\(x \\in A\\) and \\(x \\notin B\\), which is equivalent to \\(x \\in A\\) and \\(x \\in \\bar{B}\\), that is, \\(x \\in A \\cap \\bar{B}\\). \\(\\square\\)\nExample If \\(U = \\{1,2,3,4,5\\}\\), \\(A = \\{1,2,3\\}\\), and \\(B = \\{2,3,4\\}\\), then \\(\\bar{B} = \\{1,5\\}\\), and: $$A - B = \\{1\\} = A \\cap \\bar{B} = \\{1,2,3\\} \\cap \\{1,5\\} = \\{1\\}$$$$A - (B \\cup C) = (A - B) \\cap (A - C)$$$$A - (B \\cap C) = (A - B) \\cup (A - C)$$If \\(A \\cap B = \\emptyset\\): \\(A - B = A\\) and \\(B - A = B\\).\nSummary Table # Property Union Intersection Commutativity \\(A \\cup B = B \\cup A\\) \\(A \\cap B = B \\cap A\\) Associativity \\((A \\cup B) \\cup C = A \\cup (B \\cup C)\\) \\((A \\cap B) \\cap C = A \\cap (B \\cap C)\\) Distributivity \\(A \\cup (B \\cap C) = (A \\cup B) \\cap (A \\cup C)\\) \\(A \\cap (B \\cup C) = (A \\cap B) \\cup (A \\cap C)\\) De Morgan \\(\\overline{A \\cup B} = \\bar{A} \\cap \\bar{B}\\) \\(\\overline{A \\cap B} = \\bar{A} \\cup \\bar{B}\\) Absorption \\(A \\cup (A \\cap B) = A\\) \\(A \\cap (A \\cup B) = A\\) Exercises # Exercise 1: Concrete operations Let \\(U = \\{0,1,2,3,4\\}\\), \\(A = \\{0,4\\}\\), \\(B = \\{0,1,2,3\\}\\), \\(C = \\{1,4\\}\\), \\(D = \\{0,1\\}\\).\nDetermine: (a) \\(A \\cup B\\) (b) \\(B \\cap C\\) (c) \\(A \\cap B\\) (d) \\(A \\cup (B \\cap C)\\) (e) \\((A \\cup B) \\cap (A \\cup C)\\)\n(f) \\(\\overline{(A \\cap B)} \\cup \\overline{(A \\cap C)}\\) (g) \\(\\overline{A \\cup B}\\) (h) \\(A - B\\) (i) \\(B - A\\) (j) \\(A \\cup (B \\cap C \\cap D)\\)\nSolutions (a) \\(A \\cup B = \\{0,1,2,3,4\\} = U\\)\n(b) \\(B \\cap C = \\{1\\}\\)\n(c) \\(A \\cap B = \\{0\\}\\)\n(d) \\(A \\cup (B \\cap C) = \\{0,4\\} \\cup \\{1\\} = \\{0,1,4\\}\\)\n(e) \\((A \\cup B) \\cap (A \\cup C) = U \\cap \\{0,1,4\\} = \\{0,1,4\\}\\). Note that it equals (d), confirming distributivity.\n(f) \\(\\overline{A \\cap B} = U - \\{0\\} = \\{1,2,3,4\\}\\); \\(\\overline{A \\cap C} = U - \\{4\\} = \\{0,1,2,3\\}\\); hence the union is \\(\\{0,1,2,3,4\\} = U\\).\n(g) \\(A \\cup B = U\\), therefore \\(\\overline{A \\cup B} = \\emptyset\\).\n(h) \\(A - B = \\{4\\}\\) (0 is in \\(B\\), 4 is not)\n(i) \\(B - A = \\{1,2,3\\}\\)\n(j) \\(B \\cap C \\cap D = \\{0,1\\} \\cap \\{1\\} = \\{1\\}\\) (intersection of \\(B\\), \\(C\\), and \\(D\\)), so \\(A \\cup \\{1\\} = \\{0,1,4\\}\\).\nExercise 2: Symmetric difference The symmetric difference \\(A \\mathbin{\\Delta} B\\) is defined as: $$A \\mathbin{\\Delta} B := (A - B) \\cup (B - A)$$Interpret this operation geometrically and determine \\(A \\mathbin{\\Delta} B\\) for \\(A = \\{0,4\\}\\) and \\(B = \\{0,1,2,3\\}\\).\nSolution The symmetric difference retains the elements that belong to exactly one of the two sets, without the intersection. Visually, in the Venn diagram, it is the area of the two circles without the overlap. In Boolean language, this corresponds to the XOR operator (or \u0026ldquo;exclusive or\u0026rdquo;, \\(\\oplus\\)); if you want to review this connection, it is worth going back to the article on Boolean Algebra.\nA useful visual analogy: git diff highlights removed lines as \\(A - B\\), added lines as \\(B - A\\), and, together, everything that changed as \\(A \\mathbin{\\Delta} B\\). It is only an analogy: diff preserves order and context.\n\\(A - B = \\{4\\}\\), \\(B - A = \\{1,2,3\\}\\), therefore \\(A \\mathbin{\\Delta} B = \\{1,2,3,4\\}\\).\nShaded region of A Δ B: elements exclusive to A and to B, excluding the intersection In the following solutions, the colors are merely illustrative, to highlight the relevant regions in each case. It is a visual aid, not a formal convention.\nExercise 3: Diagrams with three sets Describe what the Euler diagram of each situation would look like (the drawing that shows only the regions that can have elements):\n(i) \\(A \\subset B \\subset C\\)\n(ii) \\(A \\cap B = \\emptyset\\), \\(A \\cap C = \\emptyset\\), \\(B \\cap C = \\emptyset\\)\n(iii) \\(A \\subseteq B \\cup C\\)\n(iv) \\(A \\subseteq \\bar{B}\\)\n(v) \\(A \\subseteq B - C\\)\nSolutions The following five figures are Euler diagrams, and each one omits the regions that the condition forces to be empty.\n(i) Three concentric circles: \\(A\\) innermost, \\(B\\) in the middle, \\(C\\) outermost.\nThree concentric circles representing A ⊂ B ⊂ C (ii) Three completely disjoint circles inside the rectangle \\(U\\): none touches another.\nThree disjoint sets in the universe U (iii) The circle of \\(A\\) lies entirely inside the region covered by \\(B \\cup C\\). Part of \\(A\\) can be in \\(B\\), part in \\(C\\), part in the intersection, but no part of \\(A\\) lies outside \\(B \\cup C\\).\nA contained in the region covered by B ∪ C (iv) \\(A \\subseteq \\bar{B}\\) means that no element of \\(A\\) is in \\(B\\): the circles of \\(A\\) and \\(B\\) are disjoint.\nA disjoint from B (v) \\(A \\subseteq B - C\\) means that \\(A\\) is in the part of \\(B\\) that does not belong to \\(C\\): the circle of \\(A\\) is inside \\(B\\) and completely outside \\(C\\).\nA contained in the region of B outside C The same situations in Venn diagrams. In the strict sense, the eight regions of three sets are always drawn, and those that the condition forces to be empty appear hatched in the figure below. In (i), for example, \\(A \\subset B\\) empties everything that is in \\(A\\) and outside \\(B\\) (regions 1 and 5), and \\(B \\subset C\\) empties everything that is in \\(B\\) and outside \\(C\\) (regions 2 and 4). The hatching only records what must be empty: the fact that the inclusions are strict (there being an element of \\(B\\) outside \\(A\\) and of \\(C\\) outside \\(B\\)) does not appear in the drawing.\nThe five situations of Exercise 3 as Venn diagrams: all 8 regions drawn and the necessarily empty ones hatched Next Steps # With the operations mastered, the next article deals with one question: how many elements does a set formed from operations have? The Inclusion-Exclusion Principle answers precisely that, and it is the gateway to combinatorial counting.\nSee you then!\n","date":"2026-06-09","externalUrl":null,"permalink":"/2026/06/venn-diagrams-set-operations/","section":"Blog","summary":"Union, intersection, difference, and complement are the basic operations on sets, and they show up in SQL as UNION, INTERSECT, and EXCEPT. We see each one in Venn diagrams, along with De Morgan’s laws, distributivity, and the difference between a Venn diagram and an Euler diagram.","title":"Venn Diagrams and Set Operations","type":"posts"},{"content":" Every time we group objects by some common criterion (the even numbers less than 10, the users registered in a system, the nodes of a graph), we are working with sets. The concept seems trivial, but one detail separates a rigorous mathematical grouping from a vague list: the criterion must be precise enough that any object in the world can be tested against it.\nWhy Sets? # Set theory took rigorous shape in the late nineteenth century, mainly with Georg Cantor (1845 to 1918). In discrete mathematics and computer science, sets show up everywhere:\nPropositional logic: the set of truth assignments that satisfy a formula. Combinatorics: counting subsets of size \\(k\\) of a set with \\(n\\) elements. Graphs: a graph is a pair \\((V, E)\\) where \\(V\\) is the set of vertices and \\(E \\subseteq V \\times V\\) is the set of edges. Programming: collections, filters, and searches are set operations disguised as code. This article presents the fundamental concepts of set theory: what a set is, how to describe it, the relations between sets, and the power set. These concepts form the language we will use in all the other articles of this series.\nWhat a Set Is # Definition A set is a well-defined collection of objects called elements.\nThe expression well-defined is the part of the definition that demands care. It means that, given any object, we must be able to decide unambiguously whether or not it belongs to the set. Without that precision, the grouping is not mathematically useful.\nBy convention, sets are denoted by capital letters (\\(A, B, C, \\ldots\\)) and their elements by lowercase letters (\\(a, b, x, \\ldots\\)). The relation between an object and a set is called the membership relation:\n\\(x \\in A\\): \u0026ldquo;\\(x\\) belongs to \\(A\\)\u0026rdquo; \\(x \\notin A\\): \u0026ldquo;\\(x\\) does not belong to \\(A\\)\u0026rdquo; Well-defined collections and ambiguous collections # Consider the collection \\(C\\) formed by tall people.\nIf a person is \\(1.95\\) m tall, intuitively they belong to \\(C\\). If they are \\(1.50\\) m tall, intuitively they do not. If they are \\(1.75\\) m tall, the answer is already ambiguous. This collection is not well defined because the membership criterion was not specified precisely. The right way to fix this is to replace the vague description with an objective property:\n$$C = \\{x \\mid \\text{the height of } x \\text{ is greater than } 1.75\\text{ m}\\}$$Now it works: for any person in the world, we can decide whether or not they belong to \\(C\\).\nThe flowchart below summarizes the reasoning for checking whether a collection is well defined:\n\u0026#x2197; flowchart TD A[\"Membership criterion\"] --\u003e B{\"Can any object\nbe tested\nwithout\nambiguity?\"} B -- \"Yes\" --\u003e C[\"Well-defined set ✓\"] B -- \"No / Depends\" --\u003e D[\"Ambiguous collection ✗\"] D --\u003e E[\"Rewrite with an\nobjective property\"] E --\u003e A Replacing the vague description with an objective property solves the case of tall people. But precision alone does not guarantee a set, and the discovery of this changed the foundations of mathematics:\nA precise criterion is not enough: Russell\u0026rsquo;s paradox In 1901, Bertrand Russell considered the collection of all sets that do not belong to themselves: $$R = \\{X \\mid X \\notin X\\}$$ There is nothing vague about the criterion. Still, ask whether \\(R \\in R\\): if \\(R \\in R\\), then \\(R\\) satisfies the criterion \\(X \\notin X\\), that is, \\(R \\notin R\\); if \\(R \\notin R\\), then \\(R\\) satisfies the criterion and, therefore, \\(R \\in R\\). Both answers lead to a contradiction: for the object \\(R\\), there is no way to decide membership.\nThe flaw lies in accepting that any property defines a set, the principle of so-called naive set theory. In a letter from 1902, Russell showed that this principle made the logical system of Gottlob Frege contradictory. The most widely adopted way out was the axiomatic theory of Zermelo-Fraenkel, in which a property only separates elements of a set that already exists, as in \\(\\{x \\in \\mathbb{N} \\mid x \u0026lt; 10\\}\\). Describing sets within a known universe, as we will do in this series, avoids the paradox. More at Russell\u0026rsquo;s paradox.\nThe classical number sets # Before moving on, it is useful to keep in mind the number sets that appear frequently:\nSymbol Name Description \\(\\mathbb{N}\\) Naturals \\(\\{1, 2, 3, \\ldots\\}\\) \\(\\mathbb{Z}\\) Integers \\(\\{\\ldots, -1, 0, 1, \\ldots\\}\\) \\(\\mathbb{Q}\\) Rationals \\( \\{ \\frac{p}{q} \\mid p, q \\in \\mathbb{Z}, \\ q \\neq 0 \\} \\) \\(\\mathbb{R}\\) Reals all the rationals and irrationals Convention adopted in this series: \\(0 \\notin \\mathbb{N}\\) The definition of \\(\\mathbb{N}\\) is not universal. In Brazilian basic education (BNCC), zero belongs to the naturals: \\(\\mathbb{N} = \\{0, 1, 2, \\ldots\\}\\), and \\(\\mathbb{N}^\\ast = \\{1, 2, 3, \\ldots\\}\\) is used for the nonzero naturals. In Discrete Mathematics and Computer Science, many authors (Rosen, Knuth) also include zero. In this series we adopt \\(\\mathbb{N} = \\{1, 2, 3, \\ldots\\}\\) because it follows the tradition of mathematical analysis. When the convention matters for a specific result, it will be stated explicitly. For an in-depth discussion of the different conventions, see Natural number: Terminology and notation and Is 0 a natural number? (Mathematics Stack Exchange).\nThese sets are nested inside one another: \\(\\mathbb{N} \\subset \\mathbb{Z} \\subset \\mathbb{Q} \\subset \\mathbb{R}\\). The precise meaning of the symbol \\(\\subset\\) will be formalized in the section Equality and Inclusion, where a figure illustrates this hierarchy.\nExamples of membership in \\(\\mathbb{N}\\) \\(10598 \\in \\mathbb{N}\\): natural number, belongs \\(-1 \\notin \\mathbb{N}\\): negative, does not belong \\(\\frac{1}{5} \\notin \\mathbb{N}\\): non-integer rational, does not belong \\(2.5 \\notin \\mathbb{N}\\): decimal, does not belong Ways to Describe a Set # There are two ways to specify a set:\nIn the roster notation, we explicitly list all the elements:\n$$B = \\{m, t, c, v\\}$$In the set-builder notation, we define a property \\(P(x)\\) that characterizes the elements:\n$$C = \\{x \\mid P(x)\\}$$In set-builder notation, the symbol \\(\\mid\\) is read \u0026ldquo;such that\u0026rdquo;. We can also restrict the universe of search:\n$$C = \\{x \\in \\mathbb{N} \\mid x\\ \\text{is even and}\\ x \u003c 10\\} = \\{2, 4, 6, 8\\}$$ Example: same collection, two descriptions The set of integers with absolute value less than or equal to 2 can be written in two equivalent ways:\nRoster: \\(A = \\{-2, -1, 0, 1, 2\\}\\) Set-builder: \\(A = \\{x \\in \\mathbb{Z} \\mid |x| \\leq 2\\}\\) Both describe exactly the same set.\nThe empty set # An important special case is the empty set: the set that contains no elements, denoted \\(\\emptyset\\).\nEmpty set The empty set \\(\\emptyset\\) contains no elements. $$\\emptyset = \\{x \\mid x \\in \\mathbb{N},\\ x \u003e 5\\ \\text{and}\\ x \u003c 0\\}$$ (impossible condition: no natural number is both greater than 5 and less than 0)\nThe empty set is a subset of every set, and we will see why in the next section.\nApplication: set-builder notation in databases The SQL WHERE clause is exactly a set-builder description. The query SELECT * FROM users WHERE age \u0026gt;= 18 describes the set: $$\\{x \\in \\text{users} \\mid \\text{age}(x) \\geq 18\\}$$ The whole table is the universe of search; the WHERE predicate is the property \\(P(x)\\). All data filtering is a set-builder description.\nEquality and Inclusion # Two of the most important concepts in set theory are equality and inclusion.\nEquality Two sets are equal when they have exactly the same elements: $$A = B \\iff A \\text{ and } B \\text{ have the same elements}$$A useful equivalence: $$A = B \\iff A \\subseteq B \\ \\text{and}\\ B \\subseteq A$$Note that sets have no order and no repetition: \\(\\{1, 2, 3\\} = \\{3, 1, 2\\} = \\{1, 1, 2, 3\\}\\). What matters is which elements are present.\nInclusion (subset) \\(A \\subseteq B\\) (A is contained in B): every element of \\(A\\) is also an element of \\(B\\) \\(B \\supseteq A\\) (B contains A): notation equivalent to \\(A \\subseteq B\\), read from the side of the larger set \\(A \\subset B\\) (proper inclusion): \\(A \\subseteq B\\) and \\(A \\neq B\\) A note on notation: in part of the modern literature, \\(\\subset\\) is used as a synonym for \\(\\subseteq\\) (non-strict inclusion). To avoid ambiguity, some authors adopt \\(\\subsetneq\\) exclusively for proper inclusion. In this series, \\(\\subset\\) always denotes proper inclusion, as defined above. See more at Set notation: inclusion and proper inclusion (Mathematics Stack Exchange).\nEquivalent readings for \\(A \\subseteq B\\): \u0026ldquo;\\(A\\) is contained in \\(B\\)\u0026rdquo;, \u0026ldquo;\\(A\\) is a subset of \\(B\\)\u0026rdquo;, \u0026ldquo;\\(B\\) contains \\(A\\)\u0026rdquo;, \u0026ldquo;\\(B\\) is a superset of \\(A\\)\u0026rdquo;.\nProperties of inclusion:\nProperty Statement Reflexivity \\(A \\subseteq A\\) Transitivity If \\(A \\subseteq B\\) and \\(B \\subseteq C\\), then \\(A \\subseteq C\\) Inclusion of the empty set \\(\\emptyset \\subseteq A\\) for every set \\(A\\) Proper inclusion of the empty set \\(\\emptyset \\subset A\\) for every \\(A \\neq \\emptyset\\) The property \\(\\emptyset \\subseteq A\\) follows directly from the definition: for \\(\\emptyset \\not\\subseteq A\\) to hold, there would have to exist some element of \\(\\emptyset\\) that is not in \\(A\\), but \\(\\emptyset\\) has no elements, so this situation never occurs.\nThe classical number sets are a direct example of these relations: \\(\\mathbb{N} \\subset \\mathbb{Z} \\subset \\mathbb{Q} \\subset \\mathbb{R}\\). The figure below illustrates this hierarchy of inclusions:\nInclusion relations among the classical number sets: ℕ ⊂ ℤ ⊂ ℚ ⊂ ℝ Note The figure uses nested regions to represent inclusion: each set lies geometrically inside the larger one. This way of visualizing sets is usually called a Venn diagram and will be presented formally in the next article of this series, where we will also see that, strictly speaking, a drawing with only nested regions like this one has another name: Euler diagram.\nApplication: dependency resolution as inclusion Package managers (pip, apt, npm) decide whether an installation is feasible by checking whether \\(A \\subseteq B\\). A package can be installed when: $$\\text{deps}(\\text{package}) \\subseteq \\text{installed}$$ If some dependency is not in installed, the inclusion fails and the manager looks for what is missing. \u0026ldquo;Dependency resolution\u0026rdquo; is finding the smallest additional set that makes the inclusion true.\nThe Power Set # Given a set \\(A\\), we can build a new set whose elements are all the possible subsets of \\(A\\).\nPower set The power set of \\(A\\), denoted \\(\\mathcal{P}(A)\\), is the set of all subsets of \\(A\\): $$\\mathcal{P}(A) = \\{S \\mid S \\subseteq A\\}$$In Portuguese, the name is conjunto das partes (see the Wikipedia entry Power set).\nExample: Let \\(A = \\{1, 2, 3\\}\\).\n$$\\mathcal{P}(A) = \\{\\emptyset,\\ \\{1\\},\\ \\{2\\},\\ \\{3\\},\\ \\{1,2\\},\\ \\{1,3\\},\\ \\{2,3\\},\\ \\{1,2,3\\}\\}$$The diagram below is a Hasse diagram that represents the Boolean lattice of P(A) (ordered by the inclusion relation ⊆): each edge goes up from a subset to the one immediately larger (the one that adds exactly one element).\nReading the diagram from bottom to top visually reveals how these subsets nest inside one another.\n\u0026#x2197; flowchart BT E[\"∅\"] A1[\"{1}\"] A2[\"{2}\"] A3[\"{3}\"] B12[\"{1,2}\"] B13[\"{1,3}\"] B23[\"{2,3}\"] C[\"{1,2,3}\"] E --\u003e A1 E --\u003e A2 E --\u003e A3 A1 --\u003e B12 A2 --\u003e B12 A1 --\u003e B13 A3 --\u003e B13 A2 --\u003e B23 A3 --\u003e B23 B12 --\u003e C B13 --\u003e C B23 --\u003e C What Is a Boolean Lattice? In practical (and visual) terms, it is a perfectly symmetric mathematical structure. It has an absolute \u0026ldquo;bottom\u0026rdquo; (the empty set, ∅) and an absolute \u0026ldquo;top\u0026rdquo; (the complete original set, A). Between them, the elements are connected in such a way that, if you pick any two subsets, their union and intersection are also guaranteed to be inside the structure.\nExample with the diagram above: pick \\(\\{1,2\\}\\) and \\(\\{1,3\\}\\).\nThe union \\(\\{1,2\\} \\cup \\{1,3\\} = \\{1,2,3\\}\\) is the top node. The intersection \\(\\{1,2\\} \\cap \\{1,3\\} = \\{1\\}\\) is a node at the middle level. Both results appear in the diagram: no operation \u0026ldquo;leaves\u0026rdquo; the structure.\nThe cardinality (number of elements) of the power set is given by:\n$$|\\mathcal{P}(A)| = 2^{|A|}$$Here the vertical bars indicate cardinality (the number of elements of the set), not absolute value, a notation that will be explored further in the future article on cardinality.\nFor \\(A = \\{1, 2, 3\\}\\): \\(|\\mathcal{P}(A)| = 2^3 = 8\\) subsets.\nThis makes sense combinatorially: for each element of \\(A\\), we have exactly two choices, include it in the subset or not. With \\(n\\) elements and 2 choices per element, there are \\(2^n\\) possible subsets.\nThe tree below makes this concrete for \\(A = \\{1, 2, 3\\}\\): each level represents the decision about one element, and the 8 leaves are exactly the subsets of \\(\\mathcal{P}(A)\\).\n\u0026#x2197; flowchart TD R[\"A = {1, 2, 3}\"] R --\u003e|\"1 ∈ S\"| Y1[\"{1, ?, ?}\"] R --\u003e|\"1 ∉ S\"| N1[\"{_, ?, ?}\"] Y1 --\u003e|\"2 ∈ S\"| Y12[\"{1, 2, ?}\"] Y1 --\u003e|\"2 ∉ S\"| N12[\"{1, _, ?}\"] N1 --\u003e|\"2 ∈ S\"| Y02[\"{_, 2, ?}\"] N1 --\u003e|\"2 ∉ S\"| N02[\"{_, _, ?}\"] Y12 --\u003e|\"3 ∈ S\"| L123[\"{1, 2, 3}\"] Y12 --\u003e|\"3 ∉ S\"| L12[\"{1, 2}\"] N12 --\u003e|\"3 ∈ S\"| L13[\"{1, 3}\"] N12 --\u003e|\"3 ∉ S\"| L1[\"{1}\"] Y02 --\u003e|\"3 ∈ S\"| L23[\"{2, 3}\"] Y02 --\u003e|\"3 ∉ S\"| L2[\"{2}\"] N02 --\u003e|\"3 ∈ S\"| L3[\"{3}\"] N02 --\u003e|\"3 ∉ S\"| LE[\"∅\"] The tree makes the three choices one after the other. We can also see them at the same time: if each element becomes an axis (in or out), the subsets of \\(\\{1, 2, 3\\}\\) occupy the eight vertices of a cube. The figure below draws the same lattice as the Hasse diagram in three dimensions: each edge adds a single element, the color says which one, and going up from \\(\\emptyset\\) to \\(\\{1, 2, 3\\}\\) is adding one element at a time.\nThe power set of {1, 2, 3} as a cube: each axis is the choice about one element, and the color of each edge indicates the element added Careful: elements of \\(\\mathcal{P}(A)\\) are sets # A common source of confusion is mixing up membership (\\(\\in\\)) with inclusion (\\(\\subseteq\\)) when working with \\(\\mathcal{P}(A)\\). If \\(A = \\{1, 2, 3\\}\\):\n\\( \\{1 \\} \\in \\mathcal{P}(A)\\): the set \\(\\{1\\}\\) belongs to \\(\\mathcal{P}(A)\\) (it is one of its elements) \\(1 \\notin \\mathcal{P}(A)\\): the number \\(1\\) does not belong to \\(\\mathcal{P}(A)\\) (the number is not a set) \\(\\emptyset \\in \\mathcal{P}(A)\\): the empty set always belongs \\(\\{\\{1\\}\\} \\subseteq \\mathcal{P}(A)\\): the set \\(\\{\\{1\\}\\}\\) is included in \\(\\mathcal{P}(A)\\) because its only element, \\(\\{1\\}\\), belongs to \\(\\mathcal{P}(A)\\) Summary Table # The concepts and results of the article, gathered for quick reference:\nConcept Notation Result Membership \\(x \\in A\\), \\(x \\notin A\\) relation between an object and a set Roster notation \\(A = \\{a, b, c\\}\\) lists all the elements Set-builder notation \\(A = \\{x \\in U \\mid P(x)\\}\\) elements of \\(U\\) that satisfy \\(P\\) Empty set \\(\\emptyset\\) \\(\\emptyset \\subseteq A\\) for every \\(A\\) Equality \\(A = B\\) \\(A = B \\iff A \\subseteq B \\text{ and } B \\subseteq A\\) Inclusion \\(A \\subseteq B\\) every element of \\(A\\) is in \\(B\\); it is reflexive and transitive Proper inclusion \\(A \\subset B\\) \\(A \\subseteq B\\) and \\(A \\neq B\\) Power set \\(\\mathcal{P}(A) = \\{S \\mid S \\subseteq A\\}\\) \\(\\lvert \\mathcal{P}(A) \\rvert = 2^{\\lvert A \\rvert}\\) Elements of \\(\\mathcal{P}(A)\\) \\(\\{1\\} \\in \\mathcal{P}(A)\\) they are sets: \\(1 \\notin \\mathcal{P}(A)\\) and \\(\\{\\{1\\}\\} \\subseteq \\mathcal{P}(A)\\) In the last row, \\(A = \\{1, 2, 3\\}\\), as in the article\u0026rsquo;s example.\nExercises # Exercise 1: Equality of sets Determine which of the following sets are equal: $$A = \\{a, b, -1\\} \\qquad B = \\{b, a, -1\\} \\qquad C = \\{b, a, b, -1\\} \\qquad D = \\{a, -1\\}$$ Solution \\(A = B = C\\). Sets have no order and do not count repetitions: \\(A\\), \\(B\\), and \\(C\\) contain exactly the elements \\(a\\), \\(b\\), and \\(-1\\). \\(D\\), on the other hand, does not contain \\(b\\), so \\(D \\neq A\\).\nExercise 2: Sets by roster and set-builder notation Write the following sets by explicitly listing their elements:\n(i) \\(A = \\{x \\in \\mathbb{Z} \\mid -1 \\leq x \\leq 4\\}\\)\n(ii) \\(B = \\{x \\in \\mathbb{N} \\mid x \\leq \\sqrt{10}\\ \\text{or}\\ x \u0026gt; -2\\}\\)\n(iii) \\(C = \\{x \\in \\mathbb{R} \\mid 2x + 1 = 5\\}\\)\n(iv) \\(D = \\{x \\in \\mathbb{R} \\mid x^2 + 1 = 0\\}\\)\nSolutions (i) \\(A = \\{-1, 0, 1, 2, 3, 4\\}\\). These are all the integers between \\(-1\\) and \\(4\\) inclusive.\n(ii) \\(B = \\mathbb{N} = \\{1, 2, 3, \\ldots\\}\\). The condition \\(x \u0026gt; -2\\) is satisfied by every \\(x \\in \\mathbb{N}\\), so it already covers all of \\(\\mathbb{N}\\). The condition \\(x \\leq \\sqrt{10} \\approx 3.16\\) gives \\(\\{1,2,3\\} \\subset \\mathbb{N}\\). The union of the two conditions is all of \\(\\mathbb{N}\\).\n(iii) \\(C = \\{2\\}\\), since \\(2x+1=5 \\Rightarrow x=2 \\in \\mathbb{R}\\).\n(iv) \\(D = \\emptyset\\), since \\(x^2+1=0 \\Rightarrow x^2=-1\\), which has no real solution.\nExercise 3: Membership relations Determine whether each relation is true or false:\n(i) \\(\\sqrt{2} \\in \\{x \\in \\mathbb{R} \\mid x \\geq 2\\}\\)\n(ii) \\(3 \\in \\{x \\in \\mathbb{R} \\mid |x| \\leq 4\\}\\)\n(iii) \\(\\emptyset \\notin \\mathcal{P}(A)\\) where \\(A = \\{1, 2\\}\\)\n(iv) \\(\\{1\\} \\in \\{x \\in \\mathbb{R} \\mid x^2 = 1\\}\\)\n(v) \\(\\emptyset \\in \\{\\emptyset, \\{1\\}\\}\\)\nSolutions (i) False. \\(\\sqrt{2} \\approx 1.41 \u0026lt; 2\\), so it does not satisfy \\(x \\geq 2\\).\n(ii) True. \\(|3| = 3 \\leq 4\\).\n(iii) False. \\(\\mathcal{P}(A) = \\{\\emptyset, \\{1\\}, \\{2\\}, \\{1,2\\}\\}\\) and \\(\\emptyset \\in \\mathcal{P}(A)\\).\n(iv) False. \\(\\{x \\in \\mathbb{R} \\mid x^2 = 1\\} = \\{-1, 1\\}\\). The elements are the numbers \\(-1\\) and \\(1\\), not the set \\(\\{1\\}\\).\n(v) True. The empty set is explicitly listed as an element of \\(\\{\\emptyset, \\{1\\}\\}\\).\nExercise 4: Inclusion relations Determine whether each relation is true or false:\n(i) \\(\\{-2, 0\\} \\subseteq \\{x \\in \\mathbb{Z} \\mid |x| \\leq 2\\}\\)\n(ii) \\(\\{\\pi\\} \\subset \\{1, \\{\\pi\\}, a\\}\\)\n(iii) \\(\\{\\{\\pi\\}\\} \\subset \\{1, \\{\\pi\\}, a\\}\\)\n(iv) \\(\\emptyset \\not\\subseteq \\{3, 1, -7\\}\\)\n(v) \\(\\emptyset \\subseteq \\{\\emptyset, \\{1\\}\\}\\)\nSolutions (i) True. \\(\\{x \\in \\mathbb{Z} \\mid |x| \\leq 2\\} = \\{-2,-1,0,1,2\\}\\), which contains \\(-2\\) and \\(0\\).\n(ii) False. The set \\(\\{1, \\{\\pi\\}, a\\}\\) contains the set \\(\\{\\pi\\}\\) as an element, but does not directly contain \\(\\pi\\). Hence \\(\\{\\pi\\} \\not\\subset \\{1, \\{\\pi\\}, a\\}\\).\n(iii) True. \\(\\{\\pi\\}\\) is an element of \\(\\{1, \\{\\pi\\}, a\\}\\), so \\(\\{\\{\\pi\\}\\}\\) is included in that set.\n(iv) False. \\(\\emptyset \\subseteq C\\) for every set \\(C\\), without exception.\n(v) True. \\(\\emptyset\\) is a subset of every set.\nExercise 5: Power set Given \\(A = \\{x \\in \\mathbb{Z} \\mid |x| \\leq 1\\}\\), determine \\(\\mathcal{P}(A)\\).\nSolution First, \\(A = \\{-1, 0, 1\\}\\) (integers with absolute value at most 1).\n$$\\mathcal{P}(A) = \\{\\emptyset,\\ \\{-1\\},\\ \\{0\\},\\ \\{1\\},\\ \\{-1,0\\},\\ \\{-1,1\\},\\ \\{0,1\\},\\ \\{-1,0,1\\}\\}$$Since \\(|A| = 3\\), we have \\(|\\mathcal{P}(A)| = 2^3 = 8\\) subsets, all listed above.\nNext Steps # With the notions of set, membership, and inclusion established, the natural next step is to learn to operate on sets: how to combine two sets into one, how to find what they have in common, how to remove elements of one relative to another. That is the subject of the next article, where we will also see Venn diagrams, a visual tool for reasoning about sets.\nLater in the series, all this theory gains a practical dimension: we will see how the concepts of set, membership, inclusion, and operations translate directly into Python code, with the language\u0026rsquo;s set and frozenset types.\nFor an overview of discrete mathematics and the roadmap of this series, see Discrete Mathematics: Introduction and Roadmap.\nSee you next time!\n","date":"2026-06-05","externalUrl":null,"permalink":"/2026/06/sets/","section":"Blog","summary":"The collection of tall people is not a set: there is no criterion that decides, for any person, whether they belong or not. We define set, membership, roster and set-builder descriptions, equality, inclusion, and the power set.","title":"Sets: What They Are and How to Define Them Precisely","type":"posts"},{"content":"If you have been following the site lately, you have probably noticed that the articles have been devoted to more fundamental aspects of computer science. From the internal organization of components to the logical formalism that guides the design of circuits.\nIn this article, I will lay out a roadmap of the upcoming content so that I can later use it as a point of unification for the ideas. I will also give a bit more context and the reasons for the choice of topics that I have been making recently and that I intend to keep in the near future.\nRelating theory and practice # The previous articles, as I mentioned, cover fundamental aspects of computer science. However, unlike more academic approaches, I tried whenever possible to include more practical aspects. For example, with code snippets in programming languages so that the reader can understand the consequences of the more abstract aspects. Since Python is the language I am most familiar with, I usually use it to make this link between theory and practice. But I try to write in a way that lets the reader see that the language is just a tool and can apply the concepts in any other.\nIn the next articles, I will follow this practice of adding practical aspects to content that is usually approached in a fundamentally theoretical way, as if it were closed in on itself. The next target of this approach will be Discrete Mathematics.\nDiscrete mathematics # Discrete mathematics is the part of Mathematics devoted to the study of discrete objects and structures, that is, finite or countably infinite ones (discrete means that it is made up of distinct elements, disconnected from one another).\nThe problems that discrete mathematics addresses include:\nIn how many ways can we choose a valid password for a computer? What is the probability of winning the lottery? What is the shortest path between two cities for a given transportation system? How can we sort a list of integers in ascending order? In how many steps can we do this sorting? How can we design a circuit to add two integers? These questions can be grouped into three major types of problems:\nExistence problems - Is there an arrangement of objects from a given set satisfying a certain property? Counting and enumeration problems - How many arrangements (configurations) of this type exist? Optimization problems - Of all the possible configurations, which is the best according to a certain criterion? To make the study of these problems easier, we divide discrete mathematics into fields of study. Among these fields we have:\nLogic Set theory Functions Recurrence relations Combinatorics Graphs Generally speaking, discrete mathematics is used when we count objects, when we study relations between finite sets, and when we analyze processes (algorithms) involving a finite number of steps.\nSince the second half of the 20th century it has become a very important discipline of Mathematics because, in computers, information is stored and manipulated in a discrete way.\nUpcoming series on the site # Logic already has articles here on the site: Logic for programmers, Boolean Algebra and Simplifying Logic Functions. Functions do not get a series of their own: they appear as a tool throughout the others, as in the bijection that defines graph isomorphism. For the remaining fields, the upcoming articles will be grouped into four series:\nThe Language of Sets: set theory From Base Case to Infinity: mathematical induction and recurrence relations The Art of Counting: combinatorics The Mathematics of Connections: graphs The links above point to the page of each series. All four are complete, with 28 articles in total.\nThe series are not independent: some use tools from the others. The diagram below shows these dependencies. Sets and Induction do not depend on each other, and Graphs, which builds on the other three, closes the path:\n\u0026#x2197; graph LR CONJ[\"The Language of Sets\nset theory\"] IND[\"From Base Case to Infinity\ninduction and recurrence\"] CONT[\"The Art of Counting\ncombinatorics\"] GRAF[\"The Mathematics of Connections\ngraphs\"] CONJ --\u003e CONT IND --\u003e CONT CONJ --\u003e GRAF IND --\u003e GRAF CONT --\u003e GRAF Going back to the questions from the beginning, here is where each one finds its answer:\nQuestion Where to find the answer How many valid passwords exist? In The Art of Counting, starting from the counting principles What is the probability of winning the lottery? The Art of Counting provides the tool: the number of possible bets is a combination What is the shortest path between two cities? In The Mathematics of Connections, which defines path and distance in graphs How do you sort a list, and in how many steps? Outside the four series: it is a topic for algorithms How do you design a circuit that adds two integers? In Boolean Algebra, with the half adder, and in the full adder of CPU in Action Origin of the series and inspiration for the material # I have already covered here on the site a bit about my background, saying that I am pursuing a second degree, in Computer Systems at CEDERJ. Follow the previous link for details.\nOne of the courses in the program has the pompous name of Fundamentals of Algorithms for Computing. But its syllabus is purely discrete mathematics. The course was very well taught, but it lacked a bit of practical applications in everyday programming. And the material is quite old. So I decided to use the material as inspiration and build a more modern and applied view.\nThe care I took when turning my notes into articles is the same I described, in more detail, in the introduction to the Inside the Computer series. In short:\nI have no connection with CEDERJ other than as a student, and the articles are independent of the course material: they were written for anyone interested in the topic and are not meant to replace it. Nothing was copied from the slides, neither passages nor figures. The figures were recreated with Draw.io, Mermaid, LaTeX or Python, created from scratch or generated by AI (and, in that case, they carry the tool\u0026rsquo;s watermark). Some of the examples are inspired by the course material or by reference books, with adaptations; others are original. The articles go beyond the syllabus: they bring examples in Python and SQL and more recent information, because the course material is old. The criticism, on this point, is aimed at the institution, which does not invest in updating it, and not at the professors who produced it. For those who are curious, here are the course slides. And also the distance-learning activities and the exams from recent years. All of these links are from a Google Drive maintained by the program\u0026rsquo;s Student Association, which I am not part of.\nThe same Student Association made the videos from the study platform available on YouTube, in this playlist. The reason is the platform\u0026rsquo;s instability, which constantly harms students. With the videos on YouTube, the chance of them going offline drops drastically.\nIf any of these links goes offline The links above point to material hosted by third parties: Google Drive folders and a YouTube playlist, maintained by the program\u0026rsquo;s Student Association. I have no control over either of them, and content like this tends to vanish without notice. If you find a dead link, leave a comment letting me know: I will try to find another way to share the material.\nI have already written a little about CEDERJ and the issue of outdated materials in another article when I was starting a series on computer organization. I recommend reading it if you are not familiar with the institution and want more details.\nFinally, when I wanted to go deeper into some subject, I used the book Fundamentos matemáticos para a ciência da computação: matemática discreta e suas aplicações, by Judith L. Gersting, the Brazilian edition of Mathematical Structures for Computer Science. Another classic reference in the field, and one of the most widely adopted textbooks, is Discrete Mathematics and Its Applications, by Kenneth H. Rosen, which also has a Portuguese edition.\nPrerequisites # None. The articles start from zero and build each concept from the definition. Basic familiarity with natural numbers and integers is enough.\nFor those reading on the day this article was published, the first article of the first series comes out tomorrow. See you then!\n","date":"2026-06-04","externalUrl":null,"permalink":"/2026/06/discrete-mathematics-introduction-roadmap/","section":"Blog","summary":"In how many ways can you choose a valid password? What is the shortest path between two cities? These are questions of discrete mathematics. This roadmap introduces the site’s four series on the topic: sets, induction and recurrence, combinatorics, and graphs, with 28 articles.","title":"Discrete Mathematics: Introduction and Roadmap","type":"posts"},{"content":"","date":"2026-05-27","externalUrl":null,"permalink":"/categories/arquitetura-de-computadores/","section":"Categorias","summary":"","title":"Arquitetura de Computadores","type":"categories"},{"content":"","date":"2026-05-27","externalUrl":null,"permalink":"/categories/computer-architecture/","section":"Categories","summary":"","title":"Computer Architecture","type":"categories"},{"content":"What happens between pressing Enter and the program running. This series opens up the machine layer by layer: Boolean algebra and the simplification of logic circuits, the memory hierarchy and the cache, the components of the CPU and the instruction cycle, the pipeline and the buses, the representation of data and instructions, the translation of source code into machine code (with a stop at the CPython pipeline), input and output, and modern parallel architectures.\nThe articles are listed below, in the order they were written.\n","date":"2026-05-27","externalUrl":null,"permalink":"/series/inside-the-computer/","section":"Series","summary":"","title":"Inside the Computer","type":"series"},{"content":"O que acontece entre apertar Enter e o programa rodar. Esta série abre a máquina camada por camada: a álgebra booleana e a simplificação dos circuitos lógicos, a hierarquia de memória e a cache, os componentes da CPU e o ciclo de instrução, o pipeline e os barramentos, a representação de dados e de instruções, a tradução de código-fonte em código de máquina — com uma parada no pipeline do CPython —, a entrada e saída, e as arquiteturas paralelas modernas.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-05-27","externalUrl":null,"permalink":"/series/por-dentro-do-computador/","section":"Séries","summary":"","title":"Por Dentro do Computador","type":"series"},{"content":" Every digital circuit, from a simple adder to a modern processor, starts with a logic function described in a truth table. From there, the engineer needs to find the Boolean expression that implements that function with the smallest possible number of logic gates. Fewer gates mean a smaller, cheaper, and faster circuit.\nThis article is the practical companion to the previous article Boolean Algebra: the mathematical foundation of digital computing: while that article built the theoretical foundations (postulates, theorems, and identities), this one puts them to work simplifying logic functions. We present three methods: the canonical forms (minterms and maxterms), which translate any truth table directly into an algebraic expression; algebraic simplification, which applies the theorems already studied to reduce terms and literals; and the Karnaugh map, a graphical tool that makes the process visual and systematic, without depending on algebraic intuition.\nFrom the truth table to the expression: canonical forms # The two forms presented below, sum of minterms and product of maxterms, are the so-called canonical forms of a logic function: any truth table can be written directly in each of them.\nMinterms and the SOP form # Consider the majority voter for three voters A, B, C: the output \\(F\\) is 1 when two or more voters vote 1.\nA B C F 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 1 By inspection, we can write one expression for each row where \\(F = 1\\):\n$$F = \\bar{A}BC + A\\bar{B}C + AB\\bar{C} + ABC$$Each of these products contains all three variables (in direct or complemented form). These complete products have a special name:\nMinterm A minterm of \\(n\\) variables is a logical product that contains all \\(n\\) variables, each appearing exactly once, in its direct or complemented form.\nA minterm is 1 in exactly one row of the truth table. The sum of minterms (the SOP form, Sum of Products) represents the function at the points where it is 1.\nEach minterm gets an index \\(m_i\\) equal to the decimal value of the input combination: we read the values of the variables as a binary number and convert it to decimal. For example, the combination A=0, B=1, C=1 forms the binary number \\(011_2 = 3_{10}\\), so the corresponding minterm is \\(m_3\\).\nThe index does not depend on the order of the table When the truth table is built in increasing binary order (000, 001, 010, …, 111), the minterm index coincides with the row number (counting from zero). But this coincidence does not hold if the table is in another order. The index is always determined by the values of the variables, not by the position in the table.\nThe expression above can then be written in compact notation:\n$$F = \\bar{A}BC + A\\bar{B}C + AB\\bar{C} + ABC = m_3 + m_5 + m_6 + m_7$$$$\\boxed{F(A,B,C) = \\sum(3, 5, 6, 7)}$$ Maxterms and the POS form # The starting point is \\(\\bar{F}\\), the complement of \\(F\\). Since \\(F = 1\\) in combinations 3, 5, 6, and 7, then \\(\\bar{F} = 1\\) in the remaining combinations: 0, 1, 2, and 4. That is:\n$$\\bar{F} = m_0 + m_1 + m_2 + m_4$$Now we apply De Morgan to the whole expression of \\(\\bar{F}\\) to recover \\(F\\):\n$$F = \\overline{\\bar{F}} = \\overline{m_0 + m_1 + m_2 + m_4} = \\bar{m}_0 \\cdot \\bar{m}_1 \\cdot \\bar{m}_2 \\cdot \\bar{m}_4$$What is \\(\\bar{m}_i\\)? It is De Morgan applied to a minterm. For example:\n$$\\bar{m}_0 = \\overline{\\bar{A}\\bar{B}\\bar{C}} = A + B + C$$Generalizing: the negation of a minterm \\(m_i\\) is exactly the maxterm \\(M_i\\), in which the variables that were complemented become direct, and vice versa. Therefore:\n$$F = \\bar{m}_0 \\cdot \\bar{m}_1 \\cdot \\bar{m}_2 \\cdot \\bar{m}_4 = M_0 \\cdot M_1 \\cdot M_2 \\cdot M_4$$ Maxterm A maxterm of \\(n\\) variables is a logical sum that contains all \\(n\\) variables, each appearing exactly once.\nA maxterm is 0 in exactly one row of the truth table. The product of maxterms (the POS form, Product of Sums) represents the function at the points where it is 0.\nIn compact notation:\n$$\\boxed{F(A,B,C) = \\prod(0, 1, 2, 4)}$$ The SOP ↔ POS relation The minterm and maxterm indices of the same function are always complementary: if \\(F = \\sum(3,5,6,7)\\), then necessarily \\(F = \\prod(0,1,2,4)\\). The two sets add up to all \\(2^n\\) possible indices.\nWhich form to implement? In practice, the choice between SOP and POS depends both on the result of the simplification (which form has fewer terms and literals) and on the available hardware:\nNAND gates (the most common in CMOS technology) favor the SOP form: two levels of NAND gates directly implement any sum-of-products expression. NOR gates favor the POS form: two levels of NOR gates directly implement any product-of-sums expression. Therefore, it is advisable to simplify the function in both forms and compare the cost in literals before choosing which one to implement.\nGeneral table of minterms and maxterms (3 variables) # Row x y z Minterm \\(m_i\\) Maxterm \\(M_i\\) 0 0 0 0 \\(\\bar{x}\\bar{y}\\bar{z}\\) \\(x+y+z\\) 1 0 0 1 \\(\\bar{x}\\bar{y}z\\) \\(x+y+\\bar{z}\\) 2 0 1 0 \\(\\bar{x}y\\bar{z}\\) \\(x+\\bar{y}+z\\) 3 0 1 1 \\(\\bar{x}yz\\) \\(x+\\bar{y}+\\bar{z}\\) 4 1 0 0 \\(x\\bar{y}\\bar{z}\\) \\(\\bar{x}+y+z\\) 5 1 0 1 \\(x\\bar{y}z\\) \\(\\bar{x}+y+\\bar{z}\\) 6 1 1 0 \\(xy\\bar{z}\\) \\(\\bar{x}+\\bar{y}+z\\) 7 1 1 1 \\(xyz\\) \\(\\bar{x}+\\bar{y}+\\bar{z}\\) Notice the pattern: in the minterm \\(m_i\\), a variable appears complemented if the corresponding bit of \\(i\\) is 0, and in direct form if it is 1. In the maxterm \\(M_i\\) the opposite happens.\nFor example, row 5 has \\(x=1, y=0, z=1\\), that is, \\(i = 101_2 = 5_{10}\\):\n\\(m_5 = x\\bar{y}z\\): x direct (bit 1), y complemented (bit 0), z direct (bit 1) \\(M_5 = \\bar{x}+y+\\bar{z}\\): x complemented (bit 1), y direct (bit 0), z complemented (bit 1) Exercises # Exercise 1: \\(F(A,B,C) = \\sum(1,3,5,7)\\): represent it as a product of maxterms Solution The missing indices (where \\(F = 0\\)) are 0, 2, 4, 6. Therefore:\n$$F(A,B,C) = \\prod(0,2,4,6) = M_0 \\cdot M_2 \\cdot M_4 \\cdot M_6$$Expanding: $$F = (A+B+C)(A+\\bar{B}+C)(\\bar{A}+B+C)(\\bar{A}+\\bar{B}+C)$$Note that all the maxterms have \\(C\\) in direct form. This is a hint that \\(C\\) is the determining variable: \\(\\sum(1,3,5,7)\\) are exactly the odd indices, that is, all the combinations with \\(C=1\\). Algebraic simplification confirms it:\nAlgebraic simplification $$\\begin{align} F \u0026= \\bar{A}\\bar{B}C + \\bar{A}BC + A\\bar{B}C + ABC \\\\ \u0026= \\bar{A}C\\underbrace{(\\bar{B}+B)}_{=1} + AC\\underbrace{(\\bar{B}+B)}_{=1} \\\\ \u0026= \\bar{A}C + AC = C\\underbrace{(\\bar{A}+A)}_{=1} = \\boxed{C} \\end{align}$$ Exercise 2: Represent the function in the table as SOP and POS A B C F 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 0 Solution \\(F = 1\\) at indices 1, 2, 4 (combinations \\(001_2\\), \\(010_2\\), \\(100_2\\)): $$F = \\sum(1,2,4) = \\bar{A}\\bar{B}C + \\bar{A}B\\bar{C} + A\\bar{B}\\bar{C}$$\\(F = 0\\) at indices 0, 3, 5, 6, 7 (combinations \\(000_2\\), \\(011_2\\), \\(101_2\\), \\(110_2\\), \\(111_2\\)): $$F = \\prod(0,3,5,6,7) = (A+B+C)(A+\\bar{B}+\\bar{C})(\\bar{A}+B+\\bar{C})(\\bar{A}+\\bar{B}+C)(\\bar{A}+\\bar{B}+\\bar{C})$$ SOP and POS indices are always complementary Each minterm \\(m_i\\) is 1 exactly at combination \\(i\\), so the SOP lists the indices where \\(F = 1\\). Each maxterm \\(M_i\\) is 0 exactly at combination \\(i\\), so the POS lists the indices where \\(F = 0\\). Since every combination belongs to one of the two cases, \\(\\{1,2,4\\} \\cup \\{0,3,5,6,7\\} = \\{0,\\ldots,7\\}\\): the two sets partition the input space.\nAlso note the opposite complementation rule: for index 1 (\\(001_2\\)), the minterm is \\(m_1 = \\bar{A}\\bar{B}C\\) (bit 0 → complemented, bit 1 → direct), while the maxterm is \\(M_1 = A+B+\\bar{C}\\) (bit 0 → direct, bit 1 → complemented). The same bit pattern generates opposite literals in the two forms.\nAlgebraic simplification # With the expression in canonical form in hand, we can apply the postulates and theorems of Boolean algebra to reduce the number of terms and literals. The result is an equivalent but simpler expression.\nExample 1, majority voter: \\(F(A,B,C) = \\sum(3,5,6,7)\\) $$\\begin{align} F \u0026= \\bar{A}BC + A\\bar{B}C + AB\\bar{C} + ABC \\\\ \u0026= \\bar{A}BC + A\\bar{B}C + AB\\underbrace{(\\bar{C}+C)}_{=1} \\\\ \u0026= \\bar{A}BC + A\\bar{B}C + AB \\\\ \u0026= B\\underbrace{(\\bar{A}C + A)}_{\\text{Theorem: }X + \\bar{X}Y = X+Y} + A\\bar{B}C \\\\ \u0026= B(A + C) + A\\bar{B}C \\\\ \u0026= AB + BC + A\\bar{B}C \\\\ \u0026= AB + C\\underbrace{(B + A\\bar{B})}_{\\text{Theorem: }X + \\bar{X}Y = X+Y} \\\\ \u0026= AB + C(B + A) \\\\ \u0026\\boxed{F = AB + AC + BC} \\end{align}$$ Example 2: \\(F(A,B,C) = \\sum(3,7)\\) $$F = \\bar{A}BC + ABC = BC\\underbrace{(\\bar{A}+A)}_{=1} = \\boxed{BC}$$ Example 3: \\(F(A,B,C) = \\sum(4,5,6,7)\\) $$\\begin{align} F \u0026= A\\bar{B}\\bar{C} + A\\bar{B}C + AB\\bar{C} + ABC \\\\ \u0026= A\\bar{B}\\underbrace{(\\bar{C}+C)}_{=1} + AB\\underbrace{(\\bar{C}+C)}_{=1} \\\\ \u0026= A\\bar{B} + AB = A\\underbrace{(\\bar{B}+B)}_{=1} = \\boxed{A} \\end{align}$$ Example 4: \\(F(A,B,C) = \\sum(0,2,4,6)\\) $$\\begin{align} F \u0026= \\bar{A}\\bar{B}\\bar{C} + \\bar{A}B\\bar{C} + A\\bar{B}\\bar{C} + AB\\bar{C} \\\\ \u0026= \\bar{A}\\bar{C}\\underbrace{(\\bar{B}+B)}_{=1} + A\\bar{C}\\underbrace{(\\bar{B}+B)}_{=1} \\\\ \u0026= \\bar{A}\\bar{C} + A\\bar{C} = \\bar{C}\\underbrace{(\\bar{A}+A)}_{=1} = \\boxed{\\bar{C}} \\end{align}$$ Limitation of algebraic simplification The method works well for simple functions, but it is not systematic: there is no guaranteed procedure that leads to the minimum, and the sequence of steps depends on intuition and experience. For functions with many minterms or many variables, the Karnaugh map is far more efficient.\nKarnaugh map # To overcome these limitations, the Karnaugh map offers a visual and systematic procedure that leads to the minimal expression in sum of products (or in product of sums), as long as the groups are chosen correctly, without depending on algebraic intuition.\nDefinition The Karnaugh map (K-map; Maurice Karnaugh, 1953, inspired by the Veitch diagram of 1952) is a graphical representation of the truth table where each cell corresponds to a minterm. The minterms are arranged so that neighboring cells differ in exactly one variable, which lets us see simplifiable groups visually.\nThe Gray code: why the order matters # The column sequence is not binary! The columns (and rows) of the Karnaugh map follow the Gray code: each adjacent position differs in only 1 bit. For two bits, the sequence is:\n$$00 \\to 01 \\to 11 \\to 10$$Note that the order is not 00 → 01 → 10 → 11 (ordinary binary). This peculiarity is what guarantees logical adjacency between neighboring cells.\n2-variable map # For \\(F(A,B)\\), the map is a 2×2 grid:\n\\(A \\backslash B\\) \\(B=0\\) \\(B=1\\) \\(A=0\\) \\(m_0\\) \\(m_1\\) \\(A=1\\) \\(m_2\\) \\(m_3\\) Example: \\(F(A,B) = \\bar{A}B + A\\bar{B}\\)\nExample of a Karnaugh map for 2 variables The cells \\(m_1\\) and \\(m_2\\) are 1 and are diagonally opposite on the map, so they do not form any valid adjacent group. The canonical expression is already the minimal form: \\(F = A \\oplus B\\) (XOR).\n3-variable map # For \\(F(A,B,C)\\), the map is a 2×4 grid, with the columns labeled by the pair \\(BC\\) in Gray sequence:\n\\(A \\backslash BC\\) 00 01 11 10 0 \\(m_0\\) \\(m_1\\) \\(m_3\\) \\(m_2\\) 1 \\(m_4\\) \\(m_5\\) \\(m_7\\) \\(m_6\\) Example: \\(F(A,B,C) = \\bar{A}\\bar{B}\\bar{C} + \\bar{A}BC + A\\bar{B}\\bar{C} + ABC\\)\n\\(A \\backslash BC\\) 00 01 11 10 0 1 0 1 0 1 1 0 1 0 The cells \\(m_0, m_3, m_4, m_7\\) are 1, which is equivalent to \\(F(A,B,C) = \\sum(0,3,4,7)\\).\nNext, we will see how these maps let us simplify logic functions.\nSimplification rules with Karnaugh # Before looking at examples, it is worth systematizing the rules:\nGroup size Variables eliminated Variables remaining 1 cell 0 \\(n\\) 2 cells 1 \\(n-1\\) 4 cells 2 \\(n-2\\) 8 cells 3 \\(n-3\\) The simplification algorithm follows this order:\n1 Isolated cells first Minterms with no neighbor equal to 1; they do not simplify, but they are part of the function. 2 Only one possibility Minterms that can belong to only one group; that group is mandatory (an essential prime implicant). 3 Maximum number of cells For the remaining minterms, form the largest possible group (a power of 2). 4 Avoid redundancy If all the minterms of a group have already been covered by previous groups, that group generates a redundant term and must be discarded. Implicant and prime implicant Implicant of \\(F\\): any product of literals \\(P\\) such that, whenever \\(P = 1\\), also \\(F = 1\\). On the K-map, it corresponds to any valid group of 1s. Prime implicant: an implicant from which no literal can be removed without it ceasing to be an implicant, that is, the largest possible group covering certain minterms. Essential prime implicant: a prime implicant that is the only one covering at least one minterm. It must be part of the minimal cover. Groups only with powers of 2 Groups of 3, 5, 6, or any number that is not a power of 2 are invalid. The size must always be 1, 2, 4, 8, 16\u0026hellip;\nPOS simplification: group the 0s The procedure presented here, which groups the 1s, produces the SOP form (sum of products). To obtain the POS form (product of sums) directly from the K-map, we apply the same process to the 0s of the map: each group of 0s generates a simplified maxterm, and the final result is the product of these maxterms.\nSimplification: 3-variable map # Returning to the example \\(F(A,B,C) = \\sum(0,3,4,7)\\) presented above, there are two groups of adjacent cells with value 1. The following figure highlights them:\nExample of a Karnaugh map for 3 variables, with the groups highlighted Group 1: \\(m_0\\) and \\(m_4\\) (column 00, both rows) → \\(\\bar{B}\\bar{C}\\) Group 2: \\(m_3\\) and \\(m_7\\) (column 11, both rows) → \\(BC\\) Therefore, the function can be simplified to:\n$$\\boxed{F = \\bar{B}\\bar{C} + BC}$$ 4-variable map # For \\(F(A,B,C,D)\\), the map is a 4×4 grid, with rows labeled by \\(AB\\) and columns by \\(CD\\), both in Gray sequence:\n\\(AB \\backslash CD\\) 00 01 11 10 00 \\(m_0\\) \\(m_1\\) \\(m_3\\) \\(m_2\\) 01 \\(m_4\\) \\(m_5\\) \\(m_7\\) \\(m_6\\) 11 \\(m_{12}\\) \\(m_{13}\\) \\(m_{15}\\) \\(m_{14}\\) 10 \\(m_8\\) \\(m_9\\) \\(m_{11}\\) \\(m_{10}\\) The map is toroidal: the edges touch The Karnaugh map should be imagined as a torus: the first and last columns are adjacent, and the same holds for the first and last rows. This means that the four corners (\\(m_0, m_2, m_8, m_{10}\\)) also form a valid group!\nTo understand the geometry, think of folding the rectangle twice: first joining the side edges (forming a cylinder), then joining the circular edges of the cylinder (forming the torus):\nFrom the flat rectangle to the torus: first the side edges join, then the top and bottom edges. Source: Wikipedia The final result, with the minterms placed on the toroidal surface:\nToroidal representation of the Karnaugh map, with opposite edges adjacent. Source: Wikipedia Source: Karnaugh map: Wikipedia\nThe figure below shows how the function \\(F(A,B,C,D) = \\bar{A}\\bar{B}\\bar{C}D + \\bar{A}\\bar{B}C\\bar{D} + \\bar{A}\\bar{B}CD + \\bar{A}BC\\bar{D} + \\bar{A}BCD + A\\bar{B}\\bar{C}D + AB\\bar{C}\\bar{D} \\) can be simplified using the Karnaugh map:\nExample of a Karnaugh map for 4 variables, with the groups highlighted The filled map, equivalent to \\(F = \\sum(1,2,3,6,7,9,12)\\), is:\n\\(AB \\backslash CD\\) 00 01 11 10 00 0 1 1 1 01 0 0 1 1 11 1 0 0 0 10 0 1 0 0 The groups identified are:\nGroup of 4: \\(m_2, m_3, m_6, m_7\\), columns \\(CD = 10\\) and \\(CD = 11\\), rows \\(AB = 00\\) and \\(AB = 01\\); fixed variables \\(A = 0\\) and \\(C = 1\\) → \\(\\bar{A}C\\) Group of 2 (toroidal): \\(m_1, m_9\\), column \\(CD = 01\\), rows \\(AB = 00\\) and \\(AB = 10\\) (the top and bottom edges touch on the torus); fixed variables \\(B = 0, C = 0, D = 1\\) → \\(\\bar{B}\\bar{C}D\\) Isolated cell: \\(m_{12}\\), all four neighbors are 0; \\(A = 1, B = 1, C = 0, D = 0\\) → \\(AB\\bar{C}\\bar{D}\\) $$\\boxed{F = \\bar{A}C + \\bar{B}\\bar{C}D + AB\\bar{C}\\bar{D}}$$ 5-variable map # For \\(F(A,B,C,D,E)\\), the map grows to 32 cells, beyond the practical limit for manual work, but the structure follows the same principle. We use two 4×4 grids side by side: one for \\(A=0\\) and another for \\(A=1\\). In each grid, the rows are labeled by \\(BC\\) and the columns by \\(DE\\), both in Gray code (00 → 01 → 11 → 10).\nAdjacency works normally within each grid. In addition, there is cross adjacency between the two grids: a cell at position \\((i,j)\\) of the \\(A=0\\) grid is adjacent to the cell at the same position of the \\(A=1\\) grid, since the two differ only in \\(A\\). Groups can therefore extend across both grids.\nExample: \\(F(A,B,C,D,E) = \\sum(0,1,16,17,31)\\) Grid \\(A=0\\):\n\\(BC \\backslash DE\\) 00 01 11 10 00 1 1 0 0 01 0 0 0 0 11 0 0 0 0 10 0 0 0 0 Grid \\(A=1\\):\n\\(BC \\backslash DE\\) 00 01 11 10 00 1 1 0 0 01 0 0 0 0 11 0 0 1 0 10 0 0 0 0 Groups identified:\nGroup of 4 (crossed): \\(\\{m_0, m_1\\}\\) in grid \\(A=0\\) and \\(\\{m_{16}, m_{17}\\}\\) in grid \\(A=1\\), at the same position (row \\(BC=00\\), columns \\(DE=00\\) and \\(DE=01\\)). The four cells differ only in \\(A\\) and \\(E\\); what stays fixed is \\(B=0, C=0, D=0\\) → \\(\\bar{B}\\bar{C}\\bar{D}\\). Isolated cell: \\(m_{31}\\) in grid \\(A=1\\), row \\(BC=11\\), column \\(DE=11\\) → \\(A=1, B=1, C=1, D=1, E=1\\) → \\(ABCDE\\). $$\\boxed{F = \\bar{B}\\bar{C}\\bar{D} + ABCDE}$$For functions with 6 or more variables the K-map becomes impractical; at that point, the Quine-McCluskey method (algorithmic) is the natural alternative.\nDon\u0026rsquo;t-care conditions # In many real systems, some input combinations never occur or, if they do, the output does not matter. These conditions are called don\u0026rsquo;t-care conditions and are marked with X on the map.\nThe classic example is the BCD decoder (Binary-Coded Decimal): the decimal digits 0 to 9 use only the combinations 0000 to 1001. The combinations 1010 to 1111 (10 to 15) are invalid: they never appear as input. Therefore, the behavior of the output for these cells is irrelevant and can be marked as X.\nRules for using the Xs on the map Use the Xs to enlarge groups when that results in a larger group (a larger power of 2) and, therefore, a simpler term. Never form a group made only of Xs: those minterms do not need to be covered, and including them unnecessarily only adds terms. An X that does not help any group can simply be ignored. Example with don\u0026rsquo;t care Suppose \\(F(A,B,C,D) = \\sum m(1,3,5,7) + \\sum d(9,11,13,15)\\), where \\(d\\) indicates the don\u0026rsquo;t-cares.\n\\(AB \\backslash CD\\) 00 01 11 10 00 0 1 1 0 01 0 1 1 0 11 0 X X 0 10 0 X X 0 Without the Xs, minterms 1, 3, 5, 7 form a group of 4 (\\(A=0\\), \\(D=1\\)) → \\(\\bar{A}D\\).\nWith the Xs available, minterms 1, 3, 5, 7 can join 9, 11, 13, and 15 to form a group of 8 (the entire column \\(D=1\\)). Result: simply \\(\\boxed{F = D}\\).\nThe minimal solution may not be unique Depending on the grouping choices, the same Karnaugh map can generate two or more equally valid minimal expressions. This happens when a minterm can be included in different groups without changing the number of terms or literals of the final result.\nFor example, on a 3-variable map, \\(m_4\\) can be grouped either with \\(\\{m_0, m_4\\}\\) → \\(\\bar{B}\\bar{C}\\) or with \\(\\{m_4, m_6\\}\\) → \\(A\\bar{C}\\); both choices produce correct minimal expressions.\nIn practice, any minimal solution is acceptable. Logic synthesis tools choose automatically among the available alternatives.\nRedundant groups and hazards In hardware design, engineers sometimes intentionally add redundant groups to the K-map, even if they cover no new minterm, in order to eliminate hazards (glitches): spurious pulses at the output caused by the propagation delay between adjacent groups when the inputs change. This refinement goes beyond pure logic minimization and belongs to the timing analysis of combinational circuits.\nWorked exercises # Ex. 1: \\(F(A,B,C) = \\sum(0,2,4,5,6)\\) # \\(A \\backslash BC\\) 00 01 11 10 0 1 0 0 1 1 1 1 0 1 On the map below, the two groups are highlighted in color. The red group crosses the side edge, which on the torus is a neighbor of the opposite edge:\nKarnaugh map for F(A,B,C) = Σ(0,2,4,5,6) Group of 4 cells (red): \\(m_0, m_2, m_4, m_6\\) (columns 00 and 10, both rows) → \\(\\bar{C}\\) Group of 2 cells (green): \\(m_4, m_5\\) (row \\(A=1\\), columns 00 and 01) → \\(A\\bar{B}\\) $$\\boxed{F = \\bar{C} + A\\bar{B}}$$ Ex. 2: \\(F(A,B,C,D) = \\sum(3,4,5,7,9,13,14,15)\\) # \\(AB \\backslash CD\\) 00 01 11 10 00 0 0 1 0 01 1 1 1 0 11 0 1 1 1 10 0 1 0 0 The figure below highlights the four groups, each in its own color:\nKarnaugh map for F(A,B,C,D) = Σ(3,4,5,7,9,13,14,15): four essential prime implicants highlighted Identifying the essential prime implicants:\nExpression Color on the map Minterms Fixed variables Why essential \\(\\bar{A}CD\\) red \\(\\{3,7\\}\\) \\(A=0, C=1, D=1\\) only group covering \\(m_3\\) \\(\\bar{A}B\\bar{C}\\) green \\(\\{4,5\\}\\) \\(A=0, B=1, C=0\\) only group covering \\(m_4\\) \\(A\\bar{C}D\\) yellow \\(\\{9,13\\}\\) \\(A=1, C=0, D=1\\) only group covering \\(m_9\\) \\(ABC\\) cyan \\(\\{14,15\\}\\) \\(A=1, B=1, C=1\\) only group covering \\(m_{14}\\) Check: \\(\\{3,7\\} \\cup \\{4,5\\} \\cup \\{9,13\\} \\cup \\{14,15\\} = \\{3,4,5,7,9,13,14,15\\}\\) ✓\n$$\\boxed{F = \\bar{A}CD + \\bar{A}B\\bar{C} + A\\bar{C}D + ABC}$$ Ex. 3: \\(F(A,B,C,D) = \\sum(0,2,5,7,8,10,13,15)\\) # \\(AB \\backslash CD\\) 00 01 11 10 00 1 0 0 1 01 0 1 1 0 11 0 1 1 0 10 1 0 0 1 The figure below shows the two groups: the four corners, in red, and the central block, in green.\nKarnaugh map for F(A,B,C,D) = Σ(0,2,5,7,8,10,13,15): two groups of 4 that do not touch, the four corners and the central block Group of 4 (red): \\(m_0, m_2, m_8, m_{10}\\) (four corners in columns 00/10, rows 00/10) → \\(\\bar{B}\\bar{D}\\) Group of 4 (green): \\(m_5, m_7, m_{13}, m_{15}\\) → \\(BD\\) $$\\boxed{F = \\bar{B}\\bar{D} + BD = \\overline{B \\oplus D}}$$ An elegant result This is the function XNOR(B, D): the output is 1 when B and D have the same value. On the map, it appears as two groups that do not touch: the four corners, where \\(B = D = 0\\), and the central block, where \\(B = D = 1\\). Neither can grow, because changing only \\(B\\) or only \\(D\\) flips the output. This is how XOR and XNOR usually appear on a Karnaugh map: in groups that do not merge into a single term.\nEx. 4: \\(F(A,B,C,D) = \\sum(0,2,3,4,6,9,10,11,12,14)\\) # \\(AB \\backslash CD\\) 00 01 11 10 00 1 0 1 1 01 1 0 0 1 11 1 0 0 1 10 0 1 1 1 The figure below highlights the four groups. Cells that belong to more than one group appear with overlapping colors:\nKarnaugh map for F(A,B,C,D) = Σ(0,2,3,4,6,9,10,11,12,14): four groups highlighted Expression Color on the map Minterms Fixed variables Why essential \\(\\bar{A}\\bar{D}\\) red \\(\\{0,2,4,6\\}\\) \\(A=0, D=0\\) only group covering \\(m_0\\) \\(\\bar{B}C\\) green \\(\\{2,3,10,11\\}\\) \\(B=0, C=1\\) only group covering \\(m_3\\) \\(B\\bar{D}\\) yellow \\(\\{4,6,12,14\\}\\) \\(B=1, D=0\\) only group covering \\(m_{12}\\) \\(A\\bar{B}D\\) cyan \\(\\{9,11\\}\\) \\(A=1, B=0, D=1\\) only group covering \\(m_9\\) $$\\boxed{F = \\bar{A}\\bar{D} + \\bar{B}C + B\\bar{D} + A\\bar{B}D}$$ Ex. 5: \\(F(A,B,C,D) = \\sum(0,1,2,4,8,9,10,12,14)\\) # \\(AB \\backslash CD\\) 00 01 11 10 00 1 1 0 1 01 1 0 0 0 11 1 0 0 1 10 1 1 0 1 The figure below highlights the four groups, again with overlapping colors in the shared cells:\nKarnaugh map for F(A,B,C,D) = Σ(0,1,2,4,8,9,10,12,14): four groups highlighted Expression Color on the map Minterms Fixed variables Why essential \\(\\bar{B}\\bar{C}\\) red \\(\\{0,1,8,9\\}\\) \\(B=0, C=0\\) only group covering \\(m_1\\) \\(\\bar{B}\\bar{D}\\) green \\(\\{0,2,8,10\\}\\) \\(B=0, D=0\\) only group covering \\(m_2\\) \\(\\bar{C}\\bar{D}\\) yellow \\(\\{0,4,8,12\\}\\) \\(C=0, D=0\\) only group covering \\(m_4\\) \\(A\\bar{D}\\) cyan \\(\\{8,10,12,14\\}\\) \\(A=1, D=0\\) only group covering \\(m_{14}\\) In this exercise the four groups are concentrated in a few cells: \\(m_0\\) belongs to three of them and \\(m_8\\) to all four, and the overlapping colors of the previous figure are hard to tell apart. The figure below shows the same groups, one per map, with the same colors as the table:\nExercise 5 with one group per map: red (m0, m1, m8, m9), green (the four corners), yellow (the column CD = 00), and cyan (m8, m10, m12, m14), each with its expression and the minterm that only it covers $$\\boxed{F(A,B,C,D) = \\bar{B}\\bar{C} + \\bar{B}\\bar{D} + \\bar{C}\\bar{D} + A\\bar{D}}$$ Checking with Python # Solving maps by hand is the goal of this article, but the check can be automatic. The SymPy library, which has a whole series here on the site, provides in the sympy.logic module the functions SOPform and POSform: they take the list of minterms and return the minimal expression in each form, the same problem the Karnaugh map solves. The block below checks Ex. 5:\nfrom sympy import symbols from sympy.logic import SOPform, POSform A, B, C, D = symbols(\u0026#34;A B C D\u0026#34;) minterms = [0, 1, 2, 4, 8, 9, 10, 12, 14] sop = SOPform([A, B, C, D], minterms) print(sop) # (A \u0026amp; ~D) | (~B \u0026amp; ~C) | (~B \u0026amp; ~D) | (~C \u0026amp; ~D) pos = POSform([A, B, C, D], minterms) print(pos) # (~B | ~D) \u0026amp; (~C | ~D) \u0026amp; (A | ~B | ~C) # Answer obtained with the Karnaugh map map_answer = (~B \u0026amp; ~C) | (~B \u0026amp; ~D) | (~C \u0026amp; ~D) | (A \u0026amp; ~D) print(sop.equals(map_answer)) # True The SOP form is the same as the map\u0026rsquo;s, just with the terms in a different order. The novelty is the POS form: three terms and 7 literals, against four terms and 8 literals for the SOP. This is exactly the comparison that the note \u0026ldquo;Which form to implement?\u0026rdquo; recommends making before choosing the circuit. To get there with the map, it would be enough to group the 0s: \\(\\{5,7,13,15\\}\\) gives \\(\\bar{B}+\\bar{D}\\), \\(\\{3,7,11,15\\}\\) gives \\(\\bar{C}+\\bar{D}\\), and \\(\\{6,7\\}\\) gives \\(A+\\bar{B}+\\bar{C}\\). For an expression already written, instead of a list of minterms, the simplify_logic function from the same module plays the equivalent role.\nIf you prefer to check without installing anything, Charlie Coleman\u0026rsquo;s interactive Karnaugh map solver takes the list of minterms, draws the map with the groups, and returns the minimal expression in SOP or POS, all in the browser.\nComparing the methods # The two methods presented in this article (algebraic simplification and the Karnaugh map) attack the same problem with very different approaches. The table below compares them and includes the Quine-McCluskey algorithm, a third alternative for computational use that was not detailed here:\nMethod Advantage Limitation Algebraic Works with any number of variables Not systematic; depends on intuition Karnaugh Visual and intuitive; easy to identify groups Practical only up to 4 or 5 variables Quine-McCluskey Systematic and algorithmically implementable Practical only for computational implementation The Quine-McCluskey algorithm is essentially the Karnaugh map converted into a tabular procedure: it finds all the prime implicants by systematically comparing minterms and then selects the minimal cover. The central idea is to group the minterms by the number of 1s in their binary representation and compare only adjacent groups, since two minterms can combine only if they differ in exactly one bit, and this implies that the number of 1s differs by exactly one. Because it is deterministic and mechanical, it is the reference for exact minimization (the logic synthesis tools used in industry for large functions usually resort to heuristics, such as Espresso, which gives up the guarantee of an exact minimum to gain speed), but its complexity grows exponentially with the number of variables, making it impractical by hand for large functions.\nConclusion # Starting from a truth table, we learned to:\nWrite the canonical form (sum of minterms, SOP, or product of maxterms, POS) directly by inspection. Simplify algebraically, applying the theorems of Boolean algebra in sequence. Use the Karnaugh map to identify groups of adjacent minterms and obtain the minimal expression visually. The result of each simplification is a more economical digital circuit, with fewer gates and lower power consumption. This process is at the base of the design of all digital hardware.\nTo go deeper into the theorems and identities used throughout this article, see Boolean Algebra: the mathematical foundation of digital computing.\nIn the next article of the series, we leave the isolated logic gate behind and move on to the first major subsystem of the machine: the memory hierarchy. We will see how cells, addresses, and buses are organized to store data and instructions, and why every computer combines memories of such different speed, capacity, and cost.\nSee you then!\n","date":"2026-05-27","externalUrl":null,"permalink":"/2026/05/simplifying-logic-functions/","section":"Blog","summary":"How do we go from a truth table to the leanest Boolean expression? We show three paths: the canonical forms with minterms and maxterms, algebraic simplification, and the Karnaugh map, with worked exercises and a check in Python with SymPy.","title":"Simplifying Logic Functions: Minterms, Maxterms, and the Karnaugh Map","type":"posts"},{"content":" Boolean algebra is the mathematical foundation of all digital computing. Every operation a processor performs (comparing values, making decisions, storing data) can be described by combinations of just three basic logical operations. In this article, we explore this algebra from scratch: from the fundamental operations to the theorems, from De Morgan\u0026rsquo;s laws to the simplification of circuits, with practical Python examples along the way.\nThe introduction to this series reminded us that all the hardware of a computer, deep down, understands only two states: zero or one. Before opening up the memory and the processor, we start there: with the mathematics that manipulates these two values and with which all the circuits of the machine are described.\nHistorical context # In 1854, the English mathematician George Boole published An Investigation of the Laws of Thought, in which he proposed an algebraic system to represent and manipulate logical propositions using the values 0 (false) and 1 (true). Boole did not have computers in mind: he was formalizing logical reasoning in mathematical terms.\nAlmost a century later, in 1937, Claude Shannon realized that Boolean algebra could model electrical switching circuits. In his convention, which measures the impedance of the circuit, an open circuit corresponds to 1 and a closed one to 0, and so a series connection is a sum and a parallel connection is a product. In this article we use the opposite convention, which is more intuitive for switches and lamps: 1 is a closed circuit (the current flows and the lamp lights up) and 0 is an open circuit, so that series becomes AND and parallel becomes OR. With that, Boolean algebra came to be used to design circuits.\nThe idea is in the master\u0026rsquo;s thesis that Shannon wrote at MIT, A Symbolic Analysis of Relay and Switching Circuits, available in the institute\u0026rsquo;s open repository and summarized in a paper published in 1938. It is often remembered as the most important master\u0026rsquo;s thesis of the 20th century: the psychologist Howard Gardner described it as \u0026ldquo;possibly the most important, and also the most famous\u0026rdquo; of the century.\nConnection with mathematical logic If you have studied propositional logic, you will recognize the operations. The difference is in the emphasis: while propositional logic focuses on formal reasoning, Boolean algebra focuses on algebraic manipulation and implementation in circuits. See the article Logic for programmers for an introduction to propositional logic.\nVariables and logical functions # A Boolean variable (or logical variable) can only take two values: 0 (false) and 1 (true). A Boolean function takes one or more Boolean variables as input and produces a Boolean output.\nThe same logical function can be represented in four equivalent ways:\nAlgebraic expression: compact notation using mathematical operators Truth table: lists all \\(2^n\\) possible combinations of \\(n\\) variables and the corresponding output Logic symbol (logic gate): graphical representation used in circuit diagrams Equivalent electrical circuit: implementation with switches and components Size of the truth table For a function with \\(n\\) variables, the truth table has \\(2^n\\) rows. With 2 variables: 4 rows. With 3 variables: 8 rows. With 10 variables: 1,024 rows.\nExample: the function \\(F(A,B) = \\bar{A} \\cdot B + A \\cdot \\bar{B}\\) can be represented algebraically, as we have just seen, in a truth table:\nA B \\(\\bar{A} \\cdot B + A \\cdot \\bar{B}\\) 0 0 0 0 1 1 1 0 1 1 1 0 Or as a logic symbol (the symbol \\(\\oplus\\), an XOR gate, which we will see shortly), or as an equivalent electrical circuit. The four representations describe exactly the same behavior.\nFundamental operations # The three basic operations of Boolean algebra are enough to describe any logical function. In fact, only two are needed: the sets \\(\\{\\text{AND}, \\text{NOT}\\}\\) and \\(\\{\\text{OR}, \\text{NOT}\\}\\) are functionally complete, that is, any other operation can be built from them. This concept will be developed in detail after De Morgan\u0026rsquo;s laws.\nAND: Logical product # $$F(A, B, C, \\ldots, N) = A \\cdot B \\cdot C \\cdots N$$\\(F = 1\\) if and only if all inputs are 1. For two variables:\nA B \\(F = A \\cdot B\\) 0 0 0 0 1 0 1 0 0 1 1 1 For three variables, the truth table has \\(2^3 = 8\\) rows:\nA B C \\(F = A \\cdot B \\cdot C\\) 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 0 1 0 0 0 1 0 1 0 1 1 0 0 1 1 1 1 Only the last row, when all inputs are 1, results in 1.\nThe figure below shows the graphical symbol of the AND gate, used in digital circuit diagrams:\nGraphical symbol of the AND gate Electrical analogy: AND is equivalent to switches in series, and the lamp only lights up if all the switches are closed, since the current has to pass through each one of them. The figure below illustrates this concept:\nCircuit diagram equivalent to an AND gate In Python, the AND operation exists in two forms:\n# Logical operator (for booleans, evaluates with short-circuiting) True and True # True True and False # False False and True # False ← does not even evaluate the second operand # Bitwise operator (for integers, operates bit by bit) 0b1010 \u0026amp; 0b1100 # 0b1000 = 8 5 \u0026amp; 3 # 1 (0101 \u0026amp; 0011 = 0001) and vs \u0026amp; in Python and is the logical operator: it returns one of the operands (not necessarily True/False) and uses short-circuit evaluation. \u0026amp; is the bitwise operator: it operates on each pair of corresponding bits of two integers.\nWe have already covered short-circuit evaluation here before. Likewise, we have seen the bitwise operators in another article.\nOR: Logical sum # $$F(A, B, C, \\ldots, N) = A + B + C + \\cdots + N$$\\(F = 1\\) if at least one input is 1. For two variables:\nA B \\(F = A + B\\) 0 0 0 0 1 1 1 0 1 1 1 1 For three variables:\nA B C \\(F = A + B + C\\) 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 1 Only the first row, when all inputs are 0, results in 0.\nThe figure below shows the graphical symbol of the OR gate, used in digital circuit diagrams:\nGraphical symbol of the OR gate Electrical analogy: OR is equivalent to switches in parallel, and the lamp lights up if any switch is closed. The figure below illustrates this concept:\nCircuit diagram equivalent to an OR gate In Python, the OR operation also exists in two forms:\n# Logical operator True or False # True False or False # False # Bitwise operator 0b1010 | 0b1100 # 0b1110 = 14 5 | 3 # 7 (0101 | 0011 = 0111) NOT: Logical negation # $$F(A) = \\bar{A}$$It simply inverts the input: \\(F = 0\\) if \\(A = 1\\), and \\(F = 1\\) if \\(A = 0\\).\nA \\(F = \\bar{A}\\) 0 1 1 0 Notation NOT is also written as \\(A^{\\prime}\\) or \\(\\sim A\\). In the logic symbol, the small circle (bubble) at the output indicates inversion.\nThe figure below shows the graphical symbol of the NOT gate, used in digital circuit diagrams:\nGraphical symbol of the NOT gate In Python, inversion exists in two forms:\n# Logical operator not True # False not False # True # Bitwise operator (bit complement — result depends on the size of the type) ~5 # -6 (two\u0026#39;s complement in signed integers) ~0b0101 # ...11111010 (in two\u0026#39;s complement) # To work with positive bits, use a mask: ~5 \u0026amp; 0xFF # 250 (0b11111010) Derived functions # The next three functions are derived from the fundamental operations. The first two (NAND and NOR) are especially important for circuit manufacturing.\nNAND: NOT AND # $$F(A, B, \\ldots, N) = \\overline{A \\cdot B \\cdots N}$$\\(F = 0\\) if and only if all inputs are 1 (complement of AND).\nA B \\(F = \\overline{A \\cdot B}\\) 0 0 1 0 1 1 1 0 1 1 1 0 NAND and the semiconductor industry NAND is one of the cheapest gates to manufacture in CMOS technology (Complementary Metal-Oxide-Semiconductor, the dominant standard in the manufacturing of integrated circuits). We will see later that this has a fundamental consequence for chip design.\nThe figure below shows the graphical symbol of the NAND gate, used in digital circuit diagrams:\nGraphical symbol of the NAND gate We can define the NAND function in Python using the basic logical operations:\ndef nand(a, b): return not (a and b) nand(True, True) # False nand(True, False) # True nand(False, True) # True nand(False, False) # True NOR: NOT OR # $$F(A, B, \\ldots, N) = \\overline{A + B + \\cdots + N}$$\\(F = 1\\) if and only if all inputs are 0 (complement of OR).\nA B \\(F = \\overline{A + B}\\) 0 0 1 0 1 0 1 0 0 1 1 0 NOR and the semiconductor industry Like NAND, the NOR gate takes up minimal silicon area in CMOS. For this reason, both are widely preferred in chip manufacturing, a reason we will explore in detail later.\nThe figure below shows the graphical symbol of the NOR gate, used in digital circuit diagrams:\nGraphical symbol of the NOR gate We can define the NOR function in Python using the basic logical operations:\ndef nor(a, b): return not (a or b) nor(False, False) # True nor(True, False) # False XOR: Exclusive OR # $$F(A, B) = A \\oplus B = A \\cdot \\bar{B} + \\bar{A} \\cdot B$$\\(F = 1\\) when exactly one of the inputs is 1 (different inputs).\nA B \\(F = A \\oplus B\\) 0 0 0 0 1 1 1 0 1 1 1 0 XOR and parity XOR is 1 when the number of inputs equal to 1 is odd. It is the basis of parity detection circuits and of the binary half adder, which we build shortly.\nThe figure below shows the graphical symbol of the XOR gate, used in digital circuit diagrams:\nGraphical symbol of the XOR gate Examples in Python:\n# Bitwise XOR operator 5 ^ 3 # 6 (0101 ^ 0011 = 0110) True ^ True # False (1 ^ 1 = 0) True ^ False # True (1 ^ 0 = 1) # Interesting property: A ^ A = 0, A ^ 0 = A x = 42 x ^ x # 0 (any value XOR itself = 0) x ^ 0 # 42 (any value XOR 0 = itself) Symmetry of XOR XOR has two useful symmetry properties when manipulating expressions:\nInverting both inputs does not change the output: \\(A \\oplus B = \\bar{A} \\oplus \\bar{B}\\) Inverting only one input inverts the output: \\(A \\oplus \\bar{B} = \\overline{A \\oplus B}\\) As an exercise, try to prove these identities using the definition \\(A \\oplus B = A\\bar{B} + \\bar{A}B\\) together with De Morgan\u0026rsquo;s laws.\nComparison table of the 6 gates # The table below gathers the essential characteristics of the six gates studied:\nGate Expression Output = 1 when\u0026hellip; Universality AND \\(A \\cdot B\\) All inputs = 1 No OR \\(A + B\\) At least one input = 1 No NOT \\(\\bar{A}\\) Input = 0 No NAND \\(\\overline{A \\cdot B}\\) Not all inputs = 1 Yes NOR \\(\\overline{A + B}\\) No input = 1 Yes XOR \\(A \\oplus B\\) Different inputs No For quick reference, the figure below gathers in a single truth table the outputs of all the gates for the four combinations of \\(A\\) and \\(B\\). It includes XNOR, the complement of XOR (output 1 when the inputs are equal), which appears in Exercise 2 further on and returns in the next article:\nTruth table of the NOT, AND, OR, NAND, NOR, XOR and XNOR gates for the four combinations of A and B, with the symbol and the expression of each one; the outputs equal to 1 are highlighted in blue The diagram below highlights the derivation relationships: NAND and NOR result from applying NOT to the output of AND and OR, respectively; XOR is OR without the case in which both inputs are 1, that is, \\(A \\oplus B = (A + B) \\cdot \\overline{A \\cdot B}\\):\n\u0026#x2197; graph TD subgraph Básicas[\"Basic operations\"] AND[\"AND: A·B\nF=1 if all=1\"] OR[\"OR: A+B\nF=1 if any=1\"] NOT[\"NOT: Ā\nInverts the input\"] end subgraph Derivadas[\"Derived\"] NAND[\"NAND: NOT(A·B)\nF=0 if all=1\"] NOR[\"NOR: NOT(A+B)\nF=1 if all=0\"] XOR[\"XOR: A⊕B\nF=1 if inputs differ\"] end AND --\u003e|NOT at the output| NAND OR --\u003e|NOT at the output| NOR OR --\u003e|\"without the case A = B = 1\"| XOR First application: the half adder # With just two of the gates seen so far, we can already build the first arithmetic circuit. Adding two bits \\(A\\) and \\(B\\) produces two outputs: the sum bit, \\(S\\), and the carry, \\(C\\), which only appears when \\(1 + 1 = 10_2\\):\nA B \\(C\\) (carry) \\(S\\) (sum) 0 0 0 0 0 1 0 1 1 0 0 1 1 1 1 0 The \\(S\\) column is exactly the truth table of XOR, and the \\(C\\) column is that of AND:\n$$S = A \\oplus B \\qquad C = A \\cdot B$$This circuit is called a half adder. The figure below shows how it is assembled: the same two inputs feed an XOR, which generates the sum bit, and an AND, which generates the carry.\nHalf adder: inputs A and B feed an XOR, whose output is the sum bit S, and an AND, whose output is the carry C In Python, with the bitwise operators:\ndef half_adder(a, b): \u0026#34;\u0026#34;\u0026#34;Adds two bits and returns (carry, sum).\u0026#34;\u0026#34;\u0026#34; return a \u0026amp; b, a ^ b for a in (0, 1): for b in (0, 1): print(a, b, half_adder(a, b)) # 0 0 (0, 0) # 0 1 (0, 1) # 1 0 (0, 1) # 1 1 (1, 0) The \u0026ldquo;half\u0026rdquo; in the name comes from what is missing: the half adder has no input for the carry coming from the previous column, so it only works for the least significant bit. Putting two of them together with an OR gives the full adder, which adds three bits. It is in Computer Arithmetic: Addition and Subtraction and returns in this series inside the ALU of the hypothetical processor, where several full adders lined up add whole words. The variants used in real processors, such as the carry-lookahead adder, are covered in the entry Adder (electronics).\nPostulates of Boolean algebra # Postulates are the basic rules accepted without formal proof. They form the basis on which all theorems and properties are built.\n# Postulate AND form (·) OR form (+) 1 Binary variable \\(A \\in \\{0, 1\\}\\) \\(A \\in \\{0, 1\\}\\) 2 Constant with 0 \\(0 \\cdot 0 = 0\\) \\(0 + 0 = 0\\) 3 Constant with 1 \\(1 \\cdot 1 = 1\\) \\(1 + 1 = 1\\) 4 Mixed \\(1 \\cdot 0 = 0\\) \\(1 + 0 = 1\\) 5 Complement \\(\\bar{1} = 0\\) \\(\\bar{0} = 1\\) Note the duality: in general, swapping AND for OR and 0 for 1 (and vice versa) gives another valid postulate. This symmetry repeats throughout the system.\nTheorems # Theorems are consequences derived from the postulates:\n# Theorem AND form (·) OR form (+) Name 1 \\(A \\cdot 0 = 0\\) \\(A + 0 = A\\) Absorbing / identity element 2 \\(A \\cdot 1 = A\\) \\(A + 1 = 1\\) Identity / absorbing element 3 \\(A \\cdot A = A\\) \\(A + A = A\\) Idempotence 4 \\(A \\cdot \\bar{A} = 0\\) \\(A + \\bar{A} = 1\\) Complement 5 \\(\\bar{\\bar{A}} = A\\) \\(\\bar{\\bar{A}} = A\\) Double negation 6 \\(A(A+B) = A\\) \\(A + AB = A\\) Absorption 7 \\(A(\\bar{A}+B) = AB\\) \\(A + \\bar{A}B = A + B\\) Expanded absorption Identity and absorbing elements Note in Theorems 1 and 2 that the roles of 0 and 1 swap depending on the mathematical operation:\nIn the AND operation (\\(\\cdot\\)), 1 is the identity element (\\(A \\cdot 1 = A\\)) and 0 is the absorbing element (\\(A \\cdot 0 = 0\\), the output is forced to zero). In the OR operation (\\(+\\)), 0 is the identity element (\\(A + 0 = A\\)) and 1 is the absorbing element (\\(A + 1 = 1\\), the output is forced to one). Python check: idempotence and complement # Idempotence: A \u0026amp; A = A, A | A = A for a in (0, 1): assert (a \u0026amp; a) == a, f\u0026#34;Failed for a={a}\u0026#34; assert (a | a) == a, f\u0026#34;Failed for a={a}\u0026#34; # Complement: A \u0026amp; ~A = 0, A | ~A = 1 (for 1 bit) for a in (0, 1): complement = 1 - a # 1-bit complement assert (a \u0026amp; complement) == 0 assert (a | complement) == 1 print(\u0026#34;All theorems verified!\u0026#34;) Algebraic properties # The following properties govern how the operators behave when we combine multiple variables with one another: they are the essential tools for manipulating and simplifying Boolean expressions.\nCommutative # $$A \\cdot B = B \\cdot A \\qquad A + B = B + A$$The order of the operands does not change the result, just like in ordinary multiplication and addition.\nAssociative # $$A \\cdot (B \\cdot C) = (A \\cdot B) \\cdot C$$ $$A + (B + C) = (A + B) + C$$The grouping does not matter: we can omit parentheses in sequences of AND or of OR.\nDistributive # $$A \\cdot (B + C) = A \\cdot B + A \\cdot C$$This one is analogous to the distributivity of multiplication over addition in ordinary algebra. But Boolean algebra has a second distributive form with no equivalent in ordinary algebra:\n$$A + (B \\cdot C) = (A + B) \\cdot (A + C)$$ Proof of the distributivity of OR $$(A+B)(A+C) = AA + AC + AB + BC$$ $$= A + AC + AB + BC \\quad \\text{(idempotence: }A \\cdot A = A\\text{)}$$ $$= A(1 + C + B) + BC$$ $$= A \\cdot 1 + BC \\quad \\text{(absorbing element: }1 + X = 1\\text{)}$$ $$= A + BC \\quad \\checkmark$$ De Morgan\u0026rsquo;s laws # De Morgan\u0026rsquo;s laws are the most important theorems for manipulating and simplifying Boolean expressions. They are named after Augustus De Morgan (1806 to 1871), a British mathematician, friend and correspondent of Boole since 1842. The two published their books on logic in 1847, almost at the same time: Formal Logic, by De Morgan, and The Mathematical Analysis of Logic, by Boole. Similar relations were already known to medieval logicians, such as William of Ockham, but it was De Morgan who stated them in the language of formal logic that we use today. In Boolean notation:\nDe Morgan\u0026rsquo;s laws $$\\overline{A \\cdot B \\cdot C \\cdots N} = \\bar{A} + \\bar{B} + \\bar{C} + \\cdots + \\bar{N}$$ $$\\overline{A + B + C + \\cdots + N} = \\bar{A} \\cdot \\bar{B} \\cdot \\bar{C} \\cdots \\bar{N}$$Mnemonic rule: \u0026ldquo;Break the bar, flip the operation (· becomes +, + becomes ·), invert each variable.\u0026rdquo;\nTabular proof for two variables # First law: \\(\\overline{A \\cdot B} = \\bar{A} + \\bar{B}\\)\nA B \\(\\bar{A}\\) \\(\\bar{B}\\) \\(\\overline{A \\cdot B}\\) \\(\\bar{A} + \\bar{B}\\) 0 0 1 1 1 1 ✓ 0 1 1 0 1 1 ✓ 1 0 0 1 1 1 ✓ 1 1 0 0 0 0 ✓ In a circuit, the first law says that a NAND has the same output as an OR with both inputs inverted, as the figure below shows:\nLogic circuit representation of De Morgan\u0026rsquo;s first law Second law: \\(\\overline{A + B} = \\bar{A} \\cdot \\bar{B}\\)\nA B \\(\\bar{A}\\) \\(\\bar{B}\\) \\(\\overline{A + B}\\) \\(\\bar{A} \\cdot \\bar{B}\\) 0 0 1 1 1 1 ✓ 0 1 1 0 0 0 ✓ 1 0 0 1 0 0 ✓ 1 1 0 0 0 0 ✓ Likewise, the second law says that a NOR is equivalent to an AND with inverted inputs:\nLogic circuit representation of De Morgan\u0026rsquo;s second law Circuit equivalences # De Morgan\u0026rsquo;s laws reveal four ways to implement each gate using combinations of other gates. This is fundamental for standardizing designs using only NAND or only NOR:\nEquivalence Expression Reading 1 \\(\\overline{AB} = \\bar{A}+\\bar{B}\\) NAND(A,B) = OR with inverted inputs 2 \\(\\overline{A+B} = \\bar{A}\\cdot\\bar{B}\\) NOR(A,B) = AND with inverted inputs 3 \\(A \\cdot B = \\overline{\\bar{A}+\\bar{B}}\\) AND(A,B) = NOR with inverted inputs 4 \\(A + B = \\overline{\\bar{A}\\cdot\\bar{B}}\\) OR(A,B) = NAND with inverted inputs Equivalences 1 and 2 were already shown as equivalent logic circuits in the previous section. Equivalences 3 and 4 can be seen in the two figures below:\nAND implemented with NOR OR implemented with NAND Why NAND and NOR dominate the industry In CMOS technology, NAND and NOR gates take up less silicon area than AND and OR. With De Morgan\u0026rsquo;s equivalences, any design can be implemented using only one type of gate, which simplifies manufacturing.\n# Checking De Morgan in Python with bitwise operators def de_morgan_1(a, b): \u0026#34;\u0026#34;\u0026#34;NOT(A AND B) must equal (NOT A) OR (NOT B)\u0026#34;\u0026#34;\u0026#34; left_side = ~(a \u0026amp; b) right_side = (~a) | (~b) return left_side == right_side def de_morgan_2(a, b): \u0026#34;\u0026#34;\u0026#34;NOT(A OR B) must equal (NOT A) AND (NOT B)\u0026#34;\u0026#34;\u0026#34; left_side = ~(a | b) right_side = (~a) \u0026amp; (~b) return left_side == right_side # Check for all 1-bit values for a in (0, 1): for b in (0, 1): assert de_morgan_1(a, b), f\u0026#34;De Morgan 1 failed: a={a}, b={b}\u0026#34; assert de_morgan_2(a, b), f\u0026#34;De Morgan 2 failed: a={a}, b={b}\u0026#34; print(\u0026#34;De Morgan\u0026#39;s laws verified for 1 bit!\u0026#34;) Universality of gates (functional completeness) # The NAND and NOR gates, besides being cheap to manufacture, have a remarkable property: each of them is functionally complete on its own. This means that, with an unlimited supply of just one of these types of gate, it is possible to rebuild any other logic gate and, consequently, any digital circuit.\nIn principle, this property allows building complex chips with billions of identical components (for example, NAND gates in CMOS technology), although real chips use libraries with several types of gate.\nBuilding the basic logic with NAND # To prove the functional completeness of NAND (\\(\\overline{A \\cdot B}\\)), see how it can take the role of the three fundamental operations:\nNOT from NAND: Just tie the two inputs of the NAND gate together. By the idempotence law (\\(A \\cdot A = A\\)), the output will be the negation of the input. $$F = \\overline{A \\cdot A} = \\bar{A}$$ AND from NAND: Since NAND is an \u0026ldquo;inverted AND\u0026rdquo;, just invert its output using a second NAND configured as NOT. $$F = \\overline{(\\overline{A \\cdot B}) \\cdot (\\overline{A \\cdot B})} = A \\cdot B$$ OR from NAND: By De Morgan\u0026rsquo;s laws (\\(\\overline{A \\cdot B} = \\bar{A} + \\bar{B}\\)), we invert the inputs individually before passing them through a third NAND gate. $$F = \\overline{\\bar{A} \\cdot \\bar{B}} = \\bar{\\bar{A}} + \\bar{\\bar{B}} = A + B$$ Challenge for the reader Try to do the same exercise using only NOR gates. The reasoning is analogous, starting from the duality of Boolean algebra (presented in the Postulates section): how would you build a NOT, an OR and an AND using only the function \\(\\overline{A+B}\\)?\nSolution to the challenge: basic logic with NOR Like NAND, the NOR gate (\\(\\overline{A + B}\\)) is also functionally complete. The construction logic mirrors that of NAND, thanks to the duality of Boolean algebra:\nNOT from NOR: We tie the two inputs together. By the idempotence law (\\(A + A = A\\)), the output will be the negation of the input. $$F = \\overline{A + A} = \\bar{A}$$ OR from NOR: Since NOR is an \u0026ldquo;inverted OR\u0026rdquo;, just invert its output using a second NOR configured as NOT (as in step 1). $$F = \\overline{(\\overline{A + B}) + (\\overline{A + B})} = A + B$$ AND from NOR: By De Morgan\u0026rsquo;s second law (\\(\\overline{A + B} = \\bar{A} \\cdot \\bar{B}\\)), we invert the inputs \\(A\\) and \\(B\\) individually (using NORs as inverters) and feed the result to a third NOR gate. $$F = \\overline{\\bar{A} + \\bar{B}} = \\bar{\\bar{A}} \\cdot \\bar{\\bar{B}} = A \\cdot B$$ Simplifying logical expressions # Why simplify? # Simpler expressions correspond to circuits with fewer logic gates, which means:\nLower manufacturing cost (fewer components) Higher speed (fewer propagation delays) Lower power consumption Simplification uses the postulates and theorems of the algebra to rewrite an expression in an equivalent form, but with fewer terms or variables.\nWorked examples # The following examples progressively apply the tools seen so far, from simple properties to the Consensus Theorem, showing the algebraic reasoning step by step:\nExample 1: Distributive property of OR Show that \\(A + BC = (A + B)(A + C)\\)\n$$(A+B)(A+C) = AA + AC + AB + BC$$ $$= A + AC + AB + BC \\quad \\text{(idempotence)}$$ $$= A(1 + C + B) + BC$$ $$= A + BC \\quad \\checkmark \\quad \\text{(absorbing element: } 1 + X = 1\\text{)}$$ Example 2: Expanded absorption Show that \\(A + \\bar{A}B = A + B\\)\n$$A + \\bar{A}B = A + AB + \\bar{A}B \\quad \\text{(absorption: } A = A + AB\\text{)}$$ $$= A + B(A + \\bar{A})$$ $$= A + B \\cdot 1 = A + B \\quad \\checkmark$$ Example 3: Simplification with three variables Simplify \\(F(x,y,z) = \\bar{x}yz + x\\bar{y}z + xy\\bar{z} + xyz\\)\n$$F = \\bar{x}yz + x\\bar{y}z + xy(\\bar{z} + z)$$ $$= \\bar{x}yz + x\\bar{y}z + xy \\quad \\text{(complement: } \\bar{z} + z = 1\\text{)}$$ $$= y(\\bar{x}z + x) + x\\bar{y}z$$ $$\\text{Applying expanded absorption with } A=x, B=z: \\quad \\bar{x}z + x = x + z$$ $$= y(x + z) + x\\bar{y}z = xy + yz + x\\bar{y}z$$ $$= yz + x(y + \\bar{y}z) = yz + x(y + z) \\quad \\text{(expanded absorption again)}$$ $$\\boxed{F(x,y,z) = xy + xz + yz}$$The result \\(xy + xz + yz\\) is the majority voter function for 3 variables: the output is 1 when at least 2 of the 3 inputs are 1.\nIt is also, exactly, the carry of a full adder. Replacing \\(x, y, z\\) with the bits \\(A\\), \\(B\\) and the carry coming from the previous column, \\(C_{in}\\), we get \\(C_{out} = AB + AC_{in} + BC_{in}\\): there is a carry whenever at least two of the three bits being added are 1. The full adder appears, written as \\(AB + C_{in}(A \\oplus B)\\) (the same function), in Computer Arithmetic: Addition and Subtraction, and returns in this series inside the ALU of the hypothetical processor.\nExample 4: Consensus Theorem Show that \\(AB + \\bar{A}C + BC = AB + \\bar{A}C\\)\nThe term \\(BC\\) is redundant, because every combination that makes it 1 is already covered by the other two terms:\nWhen \\(A = 1\\): \\(BC\\) is only 1 if \\(B = C = 1\\), but \\(AB\\) already covers that case. When \\(A = 0\\): \\(BC\\) is only 1 if \\(B = C = 1\\), but \\(\\bar{A}C\\) already covers that case. Formally: $$AB + \\bar{A}C + BC = AB + \\bar{A}C + BC(A + \\bar{A})$$ $$= AB + \\bar{A}C + ABC + \\bar{A}BC$$ $$= AB(1 + C) + \\bar{A}C(1 + B)$$ $$= AB + \\bar{A}C \\quad \\checkmark$$ Example 5: Radical simplification Simplify \\(F(x,y,z) = xyz + \\bar{x}y\\bar{z} + \\bar{x}yz + xy\\bar{z}\\)\nRegrouping the terms that share \\(y\\):\n$$F = xy(z + \\bar{z}) + \\bar{x}y(\\bar{z} + z)$$ $$= xy \\cdot 1 + \\bar{x}y \\cdot 1 \\quad \\text{(complement: } z + \\bar{z} = 1\\text{)}$$ $$= y(x + \\bar{x}) = y \\cdot 1$$ $$\\boxed{F(x,y,z) = y}$$An expression with 4 terms and 12 literals reduces to a single variable. The result means that the output is 1 exactly when \\(y = 1\\), regardless of the values of \\(x\\) and \\(z\\).\nExample 6: Circuit analysis with simplification Simplify \\(F(A,B) = (A \\oplus B) + B\\)\nExpanding the XOR: \\(A \\oplus B = \\bar{A}B + A\\bar{B}\\)\n$$F = \\bar{A}B + A\\bar{B} + B$$ $$= B(\\bar{A} + 1) + A\\bar{B}$$ $$= B + A\\bar{B} \\quad \\text{(absorbing element: } 1 + X = 1\\text{)}$$Applying expanded absorption with \\(X = B\\): \\(B + \\bar{B}A = B + A\\)\n$$\\boxed{F(A,B) = A + B}$$The complex circuit (XOR + OR) simplifies to a single OR gate!\nCircuit analysis exercises # In the following exercises, we start from a circuit diagram to obtain and simplify the logical expression of the output. This skill is essential for understanding how real circuits work and how to optimize them. If you want to build the circuits and test the input combinations, CircuitVerse is a free simulator that runs in the browser; Digital Logic Sim, by Sebastian Lague, is an alternative you can install.\nExercise 1: Determine the logical expression for the output \\(F\\) The circuit to analyze is the one in the figure below:\nCircuit for the analysis of Exercise 1 It is common in this kind of exercise to identify the intermediate outputs to make the analysis easier. They have already been marked in the diagram above as \\(F1\\) and \\(F2\\).\nSolution In the circuit above, we identify the intermediate outputs:\n\\(F1 = A\\bar{B}\\) \\(F2 = \\bar{B} \\oplus C = \\bar{B} \\bar{C} + BC\\) \\(F = F1 + F2\\) $$\\boxed{F(A,B,C) = A\\bar{B} + \\bar{B} \\bar{C} + BC}$$ Exercise 2: Determine the logical expression for the output \\(F\\) The circuit to analyze is the one in the figure below:\nCircuit for the analysis of Exercise 2 Solution As in the previous exercise, we identify the intermediate outputs:\n\\(F1 = \\overline{A \\oplus B} = \\overline{\\bar{A}B + A\\bar{B}} = AB + \\bar{A}\\bar{B}\\) \\(F2 = \\bar{B}\\) \\(F = \\overline{F1 + F2} = \\overline{AB + \\bar{A}\\bar{B} + \\bar{B}} \\) Simplifying:\n$$\\begin{align*} F \u0026= \\overline{AB + \\bar{A}\\bar{B} + \\bar{B}} \\\\ \u0026= \\overline{\\bar{B}(1 + \\bar{A}) + AB} \\\\ \u0026= \\overline{\\bar{B} + AB} \\\\ \u0026= \\overline{\\bar{B} + A} \\text{ (Expanded absorption)} \\\\ \u0026= \\bar{A}B \\text{ (De Morgan)} \\end{align*}$$That is, the original expression of the circuit is equivalent to \\(F = \\bar{A}B\\), which can be implemented with a single AND and an inverter, as shown in the diagram below:\nSimplified circuit of Exercise 2 Exercise 3: Draw and simplify the circuit of \\(F = \\overline{\\bar{A}BC} + BC\\) Draw the circuit corresponding to the expression below. Make the drawing before simplifying the expression, and then simplify it to obtain a more efficient circuit.\n$$F = \\overline{\\bar{A}BC} + BC$$ Solution Before simplification, the circuit corresponding to the expression is:\nCircuit of Exercise 3 before simplification Simplifying the expression:\n$$\\begin{align*} F \u0026= \\overline{\\bar{A}BC} + BC \\\\ \u0026= A + \\bar{B} + \\bar{C} + BC \\quad \u0026\\text{(De Morgan)} \\\\ \u0026= A + \\bar{B} + C + \\bar{C} \\quad \u0026\\text{(expanded absorption: } \\bar{B} + BC = \\bar{B} + C\\text{)} \\\\ \u0026= A + \\bar{B} + 1 \\quad \u0026\\text{(complement: } C + \\bar{C} = 1\\text{)} \\\\ \u0026= 1 \\quad \u0026\\text{(absorbing element: } X + 1 = 1\\text{)} \\end{align*}$$The expression is a tautology: the output is always 1, regardless of A, B and C. Representing this in a logic circuit, we have:\nCircuit of Exercise 3 after simplification The figure above shows an inverter gate (NOT) with its input physically connected to ground (represented by the symbol of horizontal lines, which is equivalent to 0 volts). In Boolean algebra, this visually represents the postulate of negation: \\(\\bar{0} = 1\\).\nFrom a hardware engineering point of view, this is a real and indispensable circuit. In real chips (especially in CMOS technology), the inputs of logic gates that are not being used must never be left unconnected (\u0026ldquo;floating\u0026rdquo;). Loose pins act as small antennas that pick up electromagnetic noise from the environment, causing instability in the component. In addition, an input floating at an intermediate voltage can make the gate conduct current between the supply and ground, with excessive consumption and heating. By grounding the input (forcing a continuous \u0026ldquo;0\u0026rdquo;), the safety of the circuit is ensured and a constant, clean logic level \u0026ldquo;1\u0026rdquo; is obtained at the output.\nExercise 4: Obtain the simplified expression for the output \\(F\\) The circuit to analyze is the one in the figure below:\nCircuit for the analysis of Exercise 4 Solution Identifying the intermediate outputs step by step:\nStep 1:\n\\(F_1 = \\overline{A + B} = \\bar{A}\\bar{B}\\) (NOR gate) Step 2:\n\\(F_2 = \\overline{F_1 \\cdot \\bar{B} \\cdot C} = \\overline{\\bar{A}\\bar{B} \\cdot \\bar{B} \\cdot C} = \\overline{\\bar{A}\\bar{B}C}\\) (using \\(\\bar{B} \\cdot \\bar{B} = \\bar{B}\\), idempotence) By De Morgan: \\(F_2 = A + B + \\bar{C}\\) Step 3:\n\\(F = \\overline{F_1 \\cdot F_2} = \\overline{F_1} + \\overline{F_2}\\) $$\\overline{F_1} = \\overline{\\bar{A}\\bar{B}} = A + B \\qquad \\overline{F_2} = \\overline{A + B + \\bar{C}} = \\bar{A}\\bar{B}C$$$$F = (A + B) + \\bar{A}\\bar{B}C$$Applying Theorem 7 twice:\n$$A + \\bar{A}\\bar{B}C = A + \\bar{B}C \\qquad \\text{and then} \\qquad B + \\bar{B}C = B + C$$$$\\boxed{F(A,B,C) = A + B + C}$$An apparently complex circuit (NOR + NAND + NAND) is equivalent to a simple 3-input OR gate! The equivalent circuit is shown below:\nCircuit of Exercise 4 after simplification Python in practice # Beyond the isolated examples, Python offers direct ways to explore Boolean algebra.\nBitwise operators: summary # The main bitwise operators in Python and their Boolean equivalents, applied to integer values:\na, b = 0b1010, 0b1101 # 10 and 13 in decimal print(f\u0026#34;a AND b : {a \u0026amp; b:04b} = {a \u0026amp; b}\u0026#34;) # 1000 = 8 print(f\u0026#34;a OR b : {a | b:04b} = {a | b}\u0026#34;) # 1111 = 15 print(f\u0026#34;a XOR b : {a ^ b:04b} = {a ^ b}\u0026#34;) # 0111 = 7 print(f\u0026#34;NOT a : {~a}\u0026#34;) # -11 (two\u0026#39;s complement) print(f\u0026#34;NOT a (4 bits): {~a \u0026amp; 0xF:04b}\u0026#34;) # 0101 = 5 Checking properties with Python # def check_boolean(n_bits=4): \u0026#34;\u0026#34;\u0026#34;Checks the main properties for all n_bits-bit values.\u0026#34;\u0026#34;\u0026#34; limit = 2**n_bits mask = limit - 1 # e.g.: 0b1111 for 4 bits errors = [] for a in range(limit): complement_a = (~a) \u0026amp; mask # n_bits-bit complement of a # Idempotence if (a \u0026amp; a) != a: errors.append(f\u0026#34;Idem AND: a={a}\u0026#34;) if (a | a) != a: errors.append(f\u0026#34;Idem OR: a={a}\u0026#34;) # Complement if (a \u0026amp; complement_a) != 0: errors.append(f\u0026#34;Comp AND: a={a}\u0026#34;) if (a | complement_a) != mask: errors.append(f\u0026#34;Comp OR: a={a}\u0026#34;) # Double negation if ((~complement_a) \u0026amp; mask) != a: errors.append(f\u0026#34;Double neg: a={a}\u0026#34;) for b in range(limit): # Commutative if (a \u0026amp; b) != (b \u0026amp; a): errors.append(f\u0026#34;Commut AND: a={a}, b={b}\u0026#34;) if (a | b) != (b | a): errors.append(f\u0026#34;Commut OR: a={a}, b={b}\u0026#34;) # De Morgan if (~(a \u0026amp; b) \u0026amp; mask) != ((~a | ~b) \u0026amp; mask): errors.append(f\u0026#34;De Morgan 1: a={a}, b={b}\u0026#34;) if (~(a | b) \u0026amp; mask) != ((~a \u0026amp; ~b) \u0026amp; mask): errors.append(f\u0026#34;De Morgan 2: a={a}, b={b}\u0026#34;) if errors: print(\u0026#34;Errors found:\u0026#34;, errors) else: print(f\u0026#34;All properties verified for {n_bits} bits!\u0026#34;) check_boolean(4) # All properties verified for 4 bits! Practical use: bitmasks in file permissions # A classic use case of Boolean algebra in systems is permission control. In Unix/Linux systems, file permissions are stored as bits:\n# Unix file permissions: the three rwx bits of a group (owner, group or others) READ = 0b100 # 4 WRITE = 0b010 # 2 EXECUTE = 0b001 # 1 # A \u0026#34;read and execute\u0026#34; permission (no write): permission = READ | EXECUTE # 0b101 = 5 # Check whether it has read permission: has_read = bool(permission \u0026amp; READ) # True # Add write permission: permission = permission | WRITE # OR: adds the bit → 0b111 = 7 # Remove execute permission: permission = permission \u0026amp; ~EXECUTE # AND with complement: clears the bit → 0b110 = 6 # Check whether the permission is exactly \u0026#34;read only\u0026#34;: has_exactly_read = permission == READ # False: it also has write These examples show that Boolean algebra is present in everyday programming tasks: from access control to data manipulation at the bit level.\nConclusion # Boolean algebra is the language with which processors, memories and all digital circuits are designed. Mastering its operations, postulates, theorems and De Morgan\u0026rsquo;s laws is useful for:\nUnderstanding how processors execute logical and arithmetic operations Designing and optimizing digital circuits Correctly using bitwise operators in Python and other languages Understanding permission systems, network protocols and cryptography The simplification exercises show that apparently complex expressions often reduce to very simple forms, and each simplification corresponds to a circuit with fewer gates.\nIn an upcoming article, we will explore systematic simplification methods: Minterms, Maxterms and Karnaugh Maps. They are tools that allow us to find the minimal form of any logic function in a methodical way.\n","date":"2026-05-20","externalUrl":null,"permalink":"/2026/05/boolean-algebra/","section":"Blog","summary":"Every operation of a processor can be described with three logical operations: AND, OR and NOT. We go from gates and truth tables to theorems, De Morgan’s laws and the simplification of expressions, with the half adder as the first circuit and examples in Python.","title":"Boolean Algebra: the mathematical foundation of digital computing","type":"posts"},{"content":" We have reached the last article of the series Inside the Computer. Over the previous thirteen articles, we built a complete understanding of how a computer works inside: from logic gates to their simplification, from the memory hierarchy to the cache, from the CPU components to the instruction cycle, from the pipeline to the buses, from data representation to program execution, and the input and output that connects everything to the outside world.\nThis article closes the cycle with the big architectural questions: why RISC principles won in the internal design of many modern processors, why CISC remains relevant at the interface of architectures such as x86, how to classify parallel architectures with Flynn\u0026rsquo;s taxonomy, and what the modern paths are to multiply performance beyond the limits of a single core.\nRISC: the philosophy of simplicity # In the 1970s, computers were dominated by CISC architectures (Complex Instruction Set Computer): processors with hundreds of complex instructions, each one able to perform multiple operations in a single call. The idea was that powerful instructions would make programs shorter and compilers simpler.\nThree independent projects questioned that premise:\nProject Institution Resulting architecture John Cocke IBM (1975) IBM 801 John Hennessy Stanford (1981) MIPS David Patterson UC Berkeley (1980 to 1981) RISC-I and, afterward, RISC-II The Berkeley project did not directly become a product, but it was the basis of one: the SPARC, launched by Sun Microsystems in 1987, was built on Berkeley\u0026rsquo;s RISC design. It is a common mistake to attribute the SPARC to Berkeley: the idea came from there, the processor came from Sun.\nThe common conclusion was empirical: analysis of real compilers showed that only a fraction of the CISC instructions was actually used. Most of the execution time involved simple operations (data movement, basic integer arithmetic, branches). Complex instructions rarely justified the cost of implementing them in hardware. It would be more efficient to have a few simple instructions, executed in one cycle, than many complex instructions, executed in several cycles.\nRISC was born (Reduced Instruction Set Computer).\nThe seven characteristics of RISC # Reduced instruction set: from a few dozen to about 150 instructions; each one simple and regular. Fixed format: all instructions have the same size (32 bits in MIPS, 32 bits in classic ARM). This greatly simplifies the decode stage of the pipeline. Load/store architecture: only LOAD and STORE access main memory; all other instructions operate exclusively on registers. Large register file: 32 or more general-purpose registers (versus 8 to 16 in x86). Low CPI on simple instructions: most instructions are designed for fast execution, ideally with a CPI (cycles per instruction) close to 1, favoring high pipeline throughput. Highly efficient pipeline: the fixed format allows parallel decoding and uniform stages. More complex compilers: the responsibility for optimizing code moves from the hardware to the compiler; the hardware gets simpler. On RISC numbers and characteristics The characteristics above describe the classic profile of RISC architectures. In real processors, there are important variations between families and generations. The central point is not an exact number of instructions or a CPI rigidly equal to 1, but the pursuit of simplicity, regularity, and pipeline efficiency. The numeric ranges are typical textbook orders of magnitude: the original RISC-I, for example, had only 31 instructions.\nRISC × CISC: the complete comparison # A classic comparison, not an absolute dogma The following table summarizes historical and didactic differences between RISC and CISC. In modern processors, several of these boundaries have become less rigid: CISC architectures use highly regular internal execution, and modern RISC architectures also incorporate complex extensions when necessary.\nCharacteristic RISC CISC Number of instructions Small (a few dozen to ~150) Large (200 to 500+) Instruction format Fixed (e.g., 32 bits in MIPS) Variable (e.g., 1 to 15 bytes in x86) Memory access Only via LOAD / STORE Any instruction can access MM Register file Many (32+) Few (8 to 16) Cycles per instruction (CPI) ~1 1 to 10+ Pipeline Simple and efficient Complex Control unit Hardwired Microprogrammed (usually) Compilers More complex Simpler Real examples MIPS, SPARC, ARM, RISC-V Intel x86, AMD64, Motorola 68000 What about power consumption? It is common to find, in tables like this one, a row saying that RISC consumes less power than CISC. The association is understandable (ARM dominates cell phones and x86 dominated desktops), but it does not hold up against the data. The reference study on the subject, Power Struggles: Revisiting the RISC vs. CISC Debate on Contemporary ARM and x86 Architectures (Blem, Menon, and Sankaralingam, HPCA 2013), measured ARM processors (Cortex-A8 and A9) and x86 processors (Atom and Core i7) on real workloads and concluded that the ISA has practically no effect on power consumption or performance. What decides it is the microarchitecture (how much pipeline, speculation, and cache the design carries) and the fabrication node.\nPhone ARM chips consume little because they were designed to consume little, not because they are RISC. Designed for the same level of performance, ARM and x86 have comparable power consumption.\nx86 is CISC on the outside, RISC on the inside Modern Intel and AMD processors are CISC externally: they maintain compatibility with decades of legacy code. But internally, they decompose CISC instructions into RISC micro-operations (µops) before executing them. This combines the compatibility of CISC with the efficiency of RISC, and it is why modern x86 processors are so fast even with an ISA that dates from the 1970s.\nReal RISC architectures # ARM (Advanced RISC Machines): dominates smartphones and embedded systems, and has also gained a lot of ground in notebooks, desktops, and the cloud. The Apple Silicon line is a striking example of ARM in personal computers. MIPS: widely used in routers, consoles (PlayStation 1/2), and as a basis for teaching computer architecture. SPARC: used in Sun/Oracle servers. It implements register windows, a technique that speeds up function calls without saving registers to memory. RISC-V: an open, royalty-free RISC architecture. It has grown strongly in research, teaching, and embedded systems, and has been advancing quickly in commercial applications as well. Intel x86 / AMD64: classic CISC that dominates desktops and servers. Variable-size instructions (1 to 15 bytes). Internally it uses RISC µops. Example: RISC × CISC, five aspects compared Aspect RISC CISC Number of instructions Small (~100), fixed format Large (200 to 300+), variable formats CPU circuits Simpler: smaller decoder, no microprogramming More complex: large decoder, microprogram in ROM Pipeline use Very efficient: fixed size makes uniform stages easy; ideally 1 instruction/cycle Less efficient: variable size complicates the pipeline Program size More lines of assembly (simple operations require more instructions) Fewer lines of assembly (each instruction does more) MM accesses Only LOAD and STORE Any instruction can access MM The RISC/CISC code size paradox deserves attention: CISC produces shorter assembly programs (fewer instructions), but each instruction takes more bits in the object code (variable format, extensive fields). RISC generates more instructions, but each one takes exactly 32 bits. Depending on the comparison (assembly vs. binary), each side can look more compact, with no contradiction.\nPerformance metrics # Comparing processors requires care: the wrong metric can lead to misleading conclusions.\nMIPS: Millions of Instructions Per Second # $$ MIPS = \\frac{f_{\\text{clock}}}{CPI \\times 10^6} $$where \\(f_{\\text{clock}}\\) is the frequency in Hz and \\(CPI\\) (Cycles Per Instruction) is the average number of cycles per instruction.\nExample: calculating MIPS without comparing different ISAs Consider two hypothetical CPUs that implement the same ISA:\nCPU A: \\(f_{\\text{clock}} = 2.0\\ \\text{GHz}\\), \\(CPI = 2\\) CPU B: \\(f_{\\text{clock}} = 1.5\\ \\text{GHz}\\), \\(CPI = 1\\) For CPU A:\n$$ MIPS_A = \\frac{2.0 \\times 10^9}{2 \\times 10^6} = 1000\\ \\text{MIPS} $$For CPU B:\n$$ MIPS_B = \\frac{1.5 \\times 10^9}{1 \\times 10^6} = 1500\\ \\text{MIPS} $$Even with a lower frequency, CPU B delivers more instructions per second because its average CPI is lower.\nLimitation of MIPS: it can be useful within a single comparison context, but it is not a reliable metric for comparing different architectures, because distinct instructions can perform very different amounts of work.\nMFLOPS and SPECmark # MFLOPS (Millions of Floating Point Operations Per Second): used for scientific computing, simulations, linear algebra, deep learning. It counts floating-point operations, not instructions. $$MFLOPS = \\frac{\\text{No. of floating-point operations}}{T_{\\text{execution}} \\times 10^6}$$ SPECmark: metric from the SPEC benchmarks (Standard Performance Evaluation Corporation), an organization that maintains standardized suites that run a representative set of real programs and compare the result with a reference system. It overcomes the limitations of MIPS by using real workloads. Flynn\u0026rsquo;s taxonomy # For decades, the industry delivered more performance in two ways: by raising the clock frequency (from megahertz to gigahertz) and by optimizing the pipeline of a single core. However, these approaches hit a physical wall: the thermal limit. Since it is no longer viable to speed up the clock of a single chip without it melting, the solution was to multiply the hardware.\nThe wall has a name and a date: the end of Dennard scaling In 1974, the same Robert Dennard who invented DRAM published the observation that sustained three decades of ever faster processors: when you shrink a transistor, the voltage and current it needs shrink in the same proportion, and the power density stays constant. Smaller transistors could, therefore, switch faster without getting hotter per area. This is Dennard scaling, and it is the engine behind the megahertz race.\nAround 2005, it stopped holding. Below a certain thickness, the leakage current of the transistors stopped shrinking along with them, and the voltage could no longer go down without the transistors ceasing to work reliably. Power density started to rise with each generation. The industry\u0026rsquo;s response was the one this article describes: stop speeding up one core and start multiplying them. That is why a processor today has 8 cores at 3 GHz, and not 1 core at 24 GHz, and the 3.8 GHz Pentium 4 from the megahertz myth was, essentially, the last of its line.\nTo understand and classify the various forms of parallelism, we usually use Flynn\u0026rsquo;s taxonomy, proposed in 1966 by Michael Flynn. It remains a useful reference for classifying most parallel architectures, based on two axes: the instruction stream and the data stream.\n\u0026#x2197; %%{init: {'flowchart': {'useMaxWidth': false}}}%% graph TD Flynn[\"Flynn's Taxonomy\n(Classification of Parallel Architectures)\"] Flynn --\u003e SISD[\"SISD\nSingle Instruction, Single Data\n─────────────────\n1 instruction, 1 data item per cycle\nClassic Von Neumann\nE.g.: single-core PC\"] Flynn --\u003e SIMD[\"SIMD\nSingle Instruction, Multiple Data\n─────────────────\n1 instruction → N data items simultaneously\nVector processing / GPU\nE.g.: Intel SSE/AVX, CUDA\"] Flynn --\u003e MISD[\"MISD\nMultiple Instruction, Single Data\n─────────────────\nN instructions → 1 data item\nRare and controversial category\nE.g.: few convincing practical cases\"] Flynn --\u003e MIMD[\"MIMD\nMultiple Instruction, Multiple Data\n─────────────────\nN instructions + N data items\nTrue, general parallelism\nE.g.: SMP, clusters, multicore\"] Type Instruction Data Description Real examples SISD Single Single Traditional Von Neumann Single-core PC SIMD Single Multiple Vector processing Intel SSE/AVX, NVIDIA GPUs, ARM NEON MISD Multiple Single Rare and controversial category Few consensus practical examples MIMD Multiple Multiple Full parallelism SMP, clusters, multicore processors The figure below arranges the four categories in the quadrant that gives meaning to the names: the rows count the instruction streams, and the columns count the data streams. In each cell, a single stream is a single line that branches out to the processing units; multiple streams are separate lines. The MISD cell appears faded for the reason given in the table: it is rare and has no consensus example.\nThe rows count the instruction streams and the columns count the data streams. In SIMD, a single line of instructions reaches all the units; in MIMD, each unit has its own instructions and its own data. Example: classifying real architectures according to Flynn Conventional single-core PC: SISD. One instruction stream operates on one data stream. Intel SSE/AVX instructions: SIMD. One vector add instruction operates on several pairs of floats simultaneously: 4 pairs with SSE (ADDPS, 128 bits), 8 with AVX (VADDPS, 256 bits), and 16 with AVX-512 (VADDPS, 512 bits). NVIDIA GPU with CUDA: usually described as SIMD/SIMT, because many elements execute the same idea of operation on distinct data, although the practical organization uses threads and execution groups specific to the GPU. Intel Core i7 processor (multicore): MIMD. Each core executes independent instructions with different data; the cores may be running completely different programs. Server cluster: MIMD. Each node runs its own program with its own data, communicating by message passing (MPI). SMP: multiple processors, one memory # SMP (Symmetric Multi-Processing) connects several identical processors to the same main memory and the same I/O system, managed by a single operating system. In the figure below, each CPU is on its own chip, with its own cache, and all of them reach memory and I/O through the same bus. The values of x in the caches illustrate the coherence problem discussed right after.\nSMP with four CPUs under a single operating system. CPU 1 changed x in its cache (Modified); CPU 2\u0026rsquo;s copy became stale (Invalid), and main memory still holds the old value. Advantages: multiple CPUs working in parallel; the failure of one CPU does not halt the system; incremental growth (add CPUs as demand requires).\nCritical problem, cache coherence: each CPU has its own caches (L1, L2). If CPU1 modifies the value of a variable in its cache, CPU2 may be reading a stale value from its cache. Coherence protocols such as MESI (Modified, Exclusive, Shared, Invalid) solve this in hardware.\nThat is the case in the figure above: in CPU 1\u0026rsquo;s cache, x has already been changed to 7 and is in state M; CPU 2\u0026rsquo;s copy still says 5, and MESI marks it I, invalid. Main memory also holds the old value: for now, the change exists only in CPU 1\u0026rsquo;s cache.\nSMP scalability bottleneck The shared bus (or, more generally, the interconnect and the shared memory) is the main bottleneck of SMP. As the number of CPUs grows, contention for access also grows. In practice, there is a scalability limit beyond which architectures such as NUMA or clusters become more suitable.\nClusters: distributed memory, unlimited scale # A cluster is a group of independent computers (nodes) connected by a high-speed network (Ethernet, InfiniBand), working in a coordinated way as a single system. Each node has its own memory; communication between nodes is done by message passing (MPI, Message Passing Interface).\nCharacteristic SMP Cluster Memory model Shared Distributed Operating system Single for all One per node Communication Bus / MM Network (Ethernet, InfiniBand) Scalability Limited (tens of sockets)¹ High (up to thousands of nodes) Cost per processor High (special hardware) Lower (standard hardware) Cache coherence Real problem (MESI protocol) Not applicable Communication latency Nanoseconds Microseconds to milliseconds ¹ The SMP limit is counted in sockets, that is, in physical processors competing for the same memory, and there it is indeed low, because the shared bus saturates. Counting cores, the number changes scale: a current two-socket server exceeds 190 cores. It is the same SMP, with the parallelism packed inside each chip.\nExample: SMP or cluster? Two practical scenarios:\nScenario A (a service that needs to scale over the years): cluster. It allows adding new nodes as demand grows (horizontal scalability). You just connect more machines to the network. SMP has scalability limited by the shared bus bottleneck.\nScenario B (very limited physical space with stable demand): SMP. A single server takes up less physical space and is managed by a single operating system. There is no need for additional network infrastructure. Ideal when no growth is expected.\nNUMA: the middle ground # NUMA (Non-Uniform Memory Access) is an architecture halfway between SMP and clusters:\nMemory is physically distributed among the nodes: each processor has local RAM. Logically, the whole address space is unified: any CPU can address any memory location. Access to local RAM: fast. Access to remote RAM (from another node): slower, because it crosses the interconnect between nodes. Used in modern high-performance servers, CC-NUMA combines physically distributed memory with a logically unified address space, preserving cache coherence in hardware.\nVector computing: SIMD in action # Vector computing applies the same operation repeatedly over large data sets (scientific arrays, simulations, linear algebra, neural networks).\nTwo implementation paths:\nMethod Description Examples ALU pipeline ALU organized in pipeline stages; elements processed sequentially with overlap Cray-1 (1976) Parallel ALUs Multiple identical ALUs process several elements simultaneously (SIMD) Intel SSE/AVX, CUDA GPUs The historical evolution:\nCray-1 (1976): the first commercially successful vector supercomputer; peak of 160 MFLOPS, impressive for the time. Intel SSE / AVX: SIMD extensions of x86. AVX-512 processes 16 32-bit floats (or 8 64-bit doubles) in a single instruction. NVIDIA GPU: thousands to tens of thousands of CUDA cores operating in massive SIMD, the SIMT architecture (Single Instruction Multiple Threads). Modern trends: multicore and many-core # Multicore # Several complete processing cores integrated into the same processor. Each core usually has private L1 and L2 caches, while higher cache levels may be shared, depending on the microarchitecture.\nThe figure below shows this arrangement in the same visual vocabulary as the SMP figure, and the difference jumps out: there, there was a chip boundary around each CPU; here there is just one, around the four cores and the L3 they share.\nTypical multicore: private L1 and L2 for each core, shared L3, and main memory off the chip. The closer to the core, the smaller and faster the memory. Examples: Intel Core and AMD Ryzen processors in desktops and notebooks, plus the Apple Silicon line in Macs. Classification: MIMD.\nSMT: the extra cores that the task manager shows # Open the task manager of a modern computer and it will probably say something like \u0026ldquo;8 cores, 16 logical processors\u0026rdquo;. The difference is simultaneous multithreading (SMT), which Intel markets as Hyper-Threading.\nThe idea starts from an observation about waste: a core spends a good part of its time waiting (for a cache miss, for a mispredicted branch), and during that wait the pipeline sits idle. With SMT, the core duplicates the registers that hold the state of a thread (PC, general-purpose registers, flags) but keeps a single copy of the rest: ALU, caches, branch predictor. Two instruction streams come to share the same execution hardware, and when one waits, the other moves forward.\nThe gain is real but modest: typically between 15% and 30% on workloads that benefit, and none on workloads that already saturate the core. A core with SMT is not equivalent to two cores: it is equivalent to one core that wastes less. That is why the operating system, when distributing tasks, prefers to occupy distinct physical cores first and only then the logical processors of the same core.\nMany-core # Many-core architectures bring together tens, hundreds, or thousands of simple execution units on the same chip, optimized for massive parallelism.\nTypical examples are modern NVIDIA and AMD GPUs. Instead of focusing on a few very complex units, these architectures prioritize high throughput for massively parallel workloads. Approximate classification: SIMD / SIMT.\nModels change; the classification stays The hardware examples in this article are cited by family (Ryzen, Core, Apple Silicon, GeForce, Radeon) and not by model, because models are replaced every year and any specific number ages fast. What does not age is the classification: ten years from now there will be processors and GPUs with other names, more cores, and more memory, and they will still be MIMD and SIMD for the same reasons described here.\nWhere each architecture fits # Architecture Flynn\u0026rsquo;s taxonomy Single-core PC SISD SSE/AVX instructions SIMD GPU (CUDA/OpenCL) SIMD / SIMT Multicore processor MIMD SMP MIMD Cluster / supercomputer MIMD The limit of multiplying cores: Amdahl\u0026rsquo;s Law # If the solution to the thermal wall was to multiply the hardware, the question remains that this article raised earlier and has not yet answered: how far does this go? Does doubling the number of cores double the performance?\nThe answer came from Gene Amdahl, who presented the argument in 1967 (the original paper contains no equations), and the formula that bears his name is the most important one about parallelism. The idea is that almost every program has a part that cannot be parallelized: reading the input, initializing structures, combining partial results, any stretch in which one step depends on the previous one. Calling \\(s\\) the fraction of time spent on that serial part, the maximum gain with \\(N\\) cores is given by Amdahl\u0026rsquo;s Law:\n$$S(N) = \\frac{1}{s + \\dfrac{1 - s}{N}}$$The parallel part, \\(1 - s\\), is divided by \\(N\\); the serial part, \\(s\\), is divided by nothing. And that is the problem. For a program with only 10% serial code (\\(s = 0.1\\)):\nCores (\\(N\\)) Gain \\(S(N)\\) 2 1.82× 4 3.08× 8 4.71× 16 6.40× 64 8.77× 1024 9.91× \\(\\infty\\) 10× With 8 cores, the program does not get 8 times faster: it gets 4.7. And no number of cores takes it beyond 10×, because as \\(N\\) grows without bound the term \\((1-s)/N\\) vanishes and what remains is \\(S = 1/s\\). The 10% serial code, which seemed like little, becomes the ceiling.\n4.71×\n8 cores\nof gain, not 8×, with 10% serial code 8.77×\n64 cores\neight times more cores, less than double the gain 10×\ninfinite cores\nthe ceiling: 1/s, imposed by the serial part Amdahl is pessimistic on purpose Amdahl\u0026rsquo;s Law assumes a fixed-size problem: the same work, distributed among more cores. In practice, whoever gets more cores usually uses them to solve bigger problems (more pixels, more data, finer simulations), and in those cases the parallel part grows while the serial part stays put. This other point of view is Gustafson\u0026rsquo;s Law (1988), and it explains why GPUs with thousands of cores are worth it. But Amdahl\u0026rsquo;s message still stands for any task of a given size: the bottleneck of parallelism is the part that cannot be parallelized, and that is where optimization effort is worth spending first.\nConclusion # We have reached the end of the series. We started from logic gates and a memory cell (the minimal arrangements of computation and storage) and arrived at processors with multiple cache levels, pipelines, DMA, hierarchical buses, and parallel architectures at several levels. Each article revealed a different layer of the same machine that responds to your command in a fraction of a second.\nThe computer you use is a work of engineering with sixty years of continuous refinement. Now you know what is inside.\nStay tuned for the site\u0026rsquo;s upcoming article series. See you next time!\n","date":"2026-05-14","externalUrl":null,"permalink":"/2026/05/advanced-architectures/","section":"Blog","summary":"With 10% serial code, eight cores make the program only 4.7 times faster, and no number of cores gets past 10 times. We close the series with RISC and CISC, performance metrics, Flynn’s taxonomy, SMP, clusters, NUMA, multicore, and Amdahl’s Law.","title":"Advanced Architectures: RISC, Flynn, and Modern Parallelism","type":"posts"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/cisc/","section":"tags","summary":"","title":"CISC","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/hardware/","section":"tags","summary":"","title":"hardware","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/multicore/","section":"tags","summary":"","title":"multicore","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/numa/","section":"tags","summary":"","title":"NUMA","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/paralelismo/","section":"tags","summary":"","title":"paralelismo","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/parallelism/","section":"tags","summary":"","title":"parallelism","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/risc/","section":"tags","summary":"","title":"RISC","type":"tags"},{"content":"","date":"2026-05-14","externalUrl":null,"permalink":"/tags/smp/","section":"tags","summary":"","title":"SMP","type":"tags"},{"content":"","date":"2026-05-12","externalUrl":null,"permalink":"/tags/dma/","section":"tags","summary":"","title":"DMA","type":"tags"},{"content":" So far in this series we have explored memory, the CPU, the pipeline, data representation, and program execution, all internal components of the computer. But a computer without input and output would be just a closed calculator: unable to receive data from the world and unable to return results. This article covers the problem that makes I/O so challenging, the three strategies for solving it, and how to calculate the impact of each one on CPU performance.\nThe fundamental I/O problem External devices are orders of magnitude slower than the CPU and main memory. The CPU operates in nanoseconds (GHz); a hard disk responds in milliseconds; a human keyboard generates events every few hundred milliseconds. The whole I/O architecture exists to manage this speed mismatch without wasting the CPU\u0026rsquo;s precious cycles.\nExternal devices: from variety to speed # The first complication of any I/O system is the diversity of devices, each with a completely different speed, protocol, and behavior:\nCategory Examples Typical transfer rate Very slow Keyboard bytes per second to a few tens of B/s Slow Mouse, printer, UART tens of B/s to KB/s Medium Hard disk (HDD), USB 2.0 MB/s to tens of MB/s Fast SATA SSD, Gigabit network hundreds of MB/s Very fast NVMe SSD, high-speed networks GB/s Rate ranges are only orders of magnitude The rates above serve as a teaching reference. In real systems, they vary widely with the interface, bus generation, controller, protocol, block size, and access pattern.\nAs for function:\nType Examples Input Keyboard, mouse, scanner, microphone, camera Output Monitor, printer, speaker Input/Output Hard disk, USB flash drive, network card, flash memory The figure below shows the general model of an I/O device, with its components and the interface with the CPU\u0026rsquo;s I/O module:\nGeneral I/O model The conversation between the I/O module and the device happens through three kinds of signal, and it is worth keeping them apart:\nControl signals tell the device what to do: send data (INPUT or READ), accept data (OUTPUT or WRITE), report its status, or perform something specific to that hardware, such as positioning a disk head. Data is the content itself: a set of bits going or coming. Status signals tell what situation the device is in. The most common pair is READY / NOT-READY, whether or not it is ready to transfer. On the device side there are two more pieces. The control logic drives the hardware in response to what the I/O module asks. And the transducer translates between worlds: on output, it converts an electrical signal into another form of energy (motion, light, sound, ink), and does the reverse on input. Since the two ends almost never work at the same pace, the transducer usually comes with a buffer, which holds the data while one end waits for the other.\nThe I/O interface: the middleman # Between the CPU and the physical device sits the I/O interface (also called the I/O controller or I/O module). In practice, the CPU does not talk to the hard disk; it talks to the hard disk\u0026rsquo;s controller. This intermediate circuit takes on five vital responsibilities to isolate the CPU from the chaos of the outside world:\nControl and timing: synchronizes the data traffic between fast internal resources and slow external devices. Communication with the CPU: decodes commands, exchanges data over the system bus, and reports the status. Communication with the device: sends electrical signals or commands specific to the peripheral\u0026rsquo;s protocol. Temporary storage (buffering): accumulates the slow bits arriving from the device until they form a word of suitable size (8, 16, 32 bits) to send to memory all at once, compensating for the speed difference. Error detection: checks for mechanical problems (e.g., paper jam) or transmission failures (corrupted bits) and notifies the CPU. To perform these tasks, every interface has at least three internal registers mapped to the CPU:\nRegister Function Data Holds the data to be transferred between CPU ↔ device Status Indicates the current condition: ready, busy, or error Control Receives commands from the CPU: start operation, select mode CPU ←————————————→ [ I/O interface ] ←————————→ Device bus [ Data reg. ] cable/protocol [ Status reg. ] [ Control reg. ] The figure below shows the block diagram of a generic I/O module. On the outside, it connects to the rest of the computer through a set of signal lines: in practice, the lines of the system bus.\nI/O block diagram On the inside, there are data registers, which hold what is in transit, and status registers, which report the current situation. In many designs a single register accumulates the status and control functions: when read, it returns the device\u0026rsquo;s situation; when written, it accepts detailed commands from the processor. That is why reading from and writing to the same address of an interface can mean very different things.\nThe internal logic talks to the processor through the control lines: the processor sends commands through them, and some of these lines come back carrying arbitration and status signals. The module also needs to recognize and generate the addresses of the devices it controls: each module has a unique address or, when it drives more than one external device, a unique set of addresses. And closing the list is logic specific to each device: the part that changes from one controller to another, and the reason a different driver exists for each piece of hardware.\nSerial × parallel transmission # Data has to travel somehow between the interface and the device. Two paradigms:\nCharacteristic Parallel Serial Bits per cycle 8, 16, or 32 simultaneously 1 bit at a time Typical distance Short (inside the case) Long (meters to km) Number of wires Many Few (2 to 4) Cost Higher Lower Examples Internal bus, LPT port USB, RS-232, Ethernet, SATA Serial transmission can be further divided into:\nSynchronous: transmitter and receiver operate with a common time reference; in the classic model, this appears as a shared clock. It is more efficient, since it avoids extra synchronization bits per character. Asynchronous: there is no explicit shared clock between the ends, and each character is delimited by special start bits (start bit) and stop bits (stop bit). This is the standard for the UART (Universal Asynchronous Receiver-Transmitter). I/O addressing: how the CPU accesses the interface # How does the CPU access the registers of the I/O interface? There are two paradigms:\nMemory-mapped I/O: the I/O registers are given addresses within the same address space used by main memory. The CPU uses the same memory access instructions (LDA, STR, MOV\u0026hellip;) to read and write I/O registers. It is simple and flexible. Drawback: part of the address map becomes occupied by I/O, and not by main memory.\nPort-mapped I/O (isolated I/O): there is a separate address space reserved for I/O: the I/O ports. It requires dedicated instructions (IN for reading, OUT for writing). It does not consume memory addresses, and the Intel x86 architecture offers this model with ports from 0x0000 to 0xFFFF.\nThe x86 uses both models, and today mostly the first It is common to see port-mapped I/O described as \u0026ldquo;the x86 model\u0026rdquo;, as opposed to memory mapping. In fact, the x86 has always had both, and the proportion between them has reversed over time.\nThe IN/OUT ports are a legacy of the original PC and remain there for compatibility: the legacy keyboard, the timer, the serial port. But every modern PCI Express device exposes its registers through MMIO, and for concrete reasons: the port space is only 64 KB and practically exhausted, while the 64-bit memory space is vast; MMIO works with any memory access instruction, taking advantage of the full set of addressing modes; and IN/OUT are instructions sensitive to the I/O privilege level (IOPL) and slow, with no pipelining or cache.\nIn other words: anyone writing a driver today, for practically any card, writes MMIO.\nThe three I/O techniques # The central problem is: how does the CPU know when the device has finished an operation and is ready to transfer data? Three strategies evolved historically, and the first of them, programmed I/O, is also the simplest to understand and the most expensive to use.\nThe figure below illustrates the three I/O techniques: Polling, Interrupt, and DMA. Each technique has a distinct interaction model between CPU, interface, and device, with clear trade-offs in terms of efficiency and overhead. They will be explored in detail below.\nThree I/O techniques Polling: programmed I/O (busy-wait) # The CPU continuously checks the device\u0026rsquo;s status register in a busy-wait loop:\nrepeat: read Status_Reg of the device if Status_Reg == READY: read/write Data_Reg exit the loop else: keep checking (does nothing useful) Advantage: extremely simple to implement in software. Drawback: busy-wait, in which the CPU wastes all of its cycles waiting, without doing any useful work. Catastrophic waste with polling In a system with a 1 GHz clock waiting for a disk with 10 ms latency: $$\\text{Wasted cycles} = 10 \\times 10^{-3}\\ \\text{s} \\times 10^9\\ \\frac{\\text{cycles}}{\\text{s}} = 10^7\\text{ cycles}$$ Ten million completely idle cycles, just waiting. In general-purpose multitasking systems, this is unacceptable.\nInterrupt: interrupt-driven I/O # The CPU starts the I/O operation and immediately returns to its work. When the device finishes, it sends an interrupt signal (IRQ, Interrupt Request). The CPU saves the current state, runs the interrupt service routine (ISR), and resumes exactly where it left off:\n\u0026#x2197; sequenceDiagram participant UCP as CPU participant Dispositivo as Device participant ISR as ISR (Interrupt Service Routine) UCP-\u003e\u003eDispositivo: Starts operation (writes to Control Reg.) UCP-\u003e\u003eUCP: Keeps running other processes Dispositivo--\u003e\u003eUCP: Raises IRQ signal when done UCP-\u003e\u003eUCP: Saves context (PC, registers) UCP-\u003e\u003eISR: Jumps to service routine ISR-\u003e\u003eISR: Reads data from Data Reg. / handles result ISR-\u003e\u003eUCP: Restores context UCP-\u003e\u003eUCP: Resumes execution where it left off Advantage: the CPU does not waste cycles waiting; it does useful work in parallel with I/O. Drawback: overhead of saving and restoring context on every interrupt; many interrupts per second can saturate the CPU. Two types of interrupt deserve attention:\nMaskable: can be temporarily disabled (CLI on the x86), useful for critical sections that must not be interrupted. Non-maskable (NMI): in general serviced even with interrupts disabled, reserved for critical hardware errors (power failure, memory error). How the CPU knows which routine to run # A question is left open in the diagram above: the interrupt signal arrives, but how does the processor find out where to jump? The keyboard and the disk cannot land in the same routine.\nThe answer is a table in memory. Each interrupt carries a vector number, and this number indexes a table that holds the address of the corresponding routine. On the x86, it is the IDT (Interrupt Descriptor Table), whose base address sits in a dedicated processor register. Receiving interrupt number 14, then, means: look up entry 14 of the table, take the address stored there, and jump to it. The operating system fills in this table during initialization, and that is how a driver \u0026ldquo;registers\u0026rdquo; a service routine.\nAnd if two interrupts arrive together? A dedicated controller decides. In old PCs it was the PIC (Programmable Interrupt Controller), the 8259A chip, with eight lines numbered by priority; in current systems it is the APIC, which exists in a local version (one per core) and an I/O version, and allows routing each interrupt to a specific core. That is why, in a multicore system, the load of handling interrupts can be distributed, or deliberately concentrated on one core, leaving the others free.\nDMA: direct memory access # The DMA controller is a dedicated circuit capable of transferring entire blocks of data between the device interface and main memory, without the CPU having to intervene for each word transferred. The CPU merely programs the operation (address, size, direction), and the DMA controller takes over moving the block over the bus.\nThe idea is older than the PC Freeing the processor from the work of moving data is not an invention of the era of home hard disks. The IBM 709, from 1958, and the 7090 that succeeded it already had I/O channels: dedicated auxiliary processors, with their own command set, that ran transfer programs while the main CPU computed. The CPU built the \u0026ldquo;channel program\u0026rdquo;, started it, and moved on; the channel signaled when it finished.\nIt is exactly the pattern of this section, half a century before it showed up as a standard in the PC. And the idea kept coming back: modern NVMe controllers operate through command queues in memory, which the device reads on its own. They are I/O channels by another name.\nThe diagram below details the steps of the DMA operation, showing how the CPU programs the controller, how the DMA steals the bus to transfer the data, and how the CPU is notified at the end of the transfer. The result is that the CPU is practically free during the entire operation, except for a small initial overhead and a final interrupt.\n\u0026#x2197; sequenceDiagram participant UCP as CPU participant DMA as DMA Controller participant MP as Main Memory participant IF as Interface / Device UCP-\u003e\u003eDMA: Programs DMA (MM address, size, direction) UCP-\u003e\u003eUCP: Keeps running other tasks DMA-\u003e\u003eUCP: Requests use of the bus UCP-\u003e\u003eDMA: Grants the bus Note over UCP,DMA: Bus handover (here in burst mode) DMA-\u003e\u003eIF: Coordinates device read/write DMA-\u003e\u003eMP: Transfers the block through memory DMA--\u003e\u003eUCP: Interrupt at the end of the transfer The diagram above details the seven steps of the DMA operation. In the first two, the CPU is still in control: it programs the DMA controller, giving it the destination address in main memory, the amount of data to transfer, and the direction of the operation (read or write). From then on, it simply keeps running other tasks: this is the fundamental gain over the earlier techniques.\nWhen the DMA controller is ready to start the transfer, it needs to use the system bus, which belongs to the CPU. So it requests the bus (step 3), and the CPU grants it (step 4), temporarily suspending its own access to it, the handover indicated by the note in the diagram. Once the bus is granted, the DMA acts autonomously: it coordinates the read or write with the device interface (step 5) and transfers the block directly to main memory (step 6), without the CPU taking part word by word. At the end, the DMA controller sends an interrupt to the CPU (step 7), signaling that the transfer has finished and that the data is already available in memory.\nIt is worth noting that the diagram is an intentional simplification: it represents the transfer as a single continuous block, hiding the fact that, in real cycle stealing, the bus request and grant repeat for each word transferred. The following note details this distinction.\nCycle stealing vs. burst mode The DMA can operate in two distinct bus control modes:\nCycle stealing: the DMA controller requests the bus, steals 1 cycle, transfers 1 word, and gives the bus back immediately. The CPU briefly resumes, and the process repeats for each word of the block. The impact on the CPU is spread out over time, but it is never completely free during the transfer.\nBurst mode: the DMA takes the bus just once and keeps control for the whole duration of the transfer. The CPU is suspended for the entire block, but the bus is used much more efficiently (without the overhead of multiple negotiations).\nThe diagram above represents, in simplified form, burst mode: the request and grant appear only once. In real cycle stealing, this negotiation repeats word by word. In both cases, the CPU does not take part in the transfer itself and is notified only at the end, via an interrupt.\nThe efficiency of this \u0026ldquo;stealing\u0026rdquo; depends on how the DMA is physically connected to the motherboard. There are three classic architectural configurations:\nSingle bus (standalone controller): the DMA and the device share the same system bus. To read data from the disk into RAM, the data travels from the Disk to the DMA and then from the DMA to the RAM. This uses the bus twice, suspending the CPU both times. Single bus (integrated controller): the DMA and the I/O interface form a single component. The data goes straight from this controller to the RAM. The CPU is suspended only once. Dedicated I/O bus: the system has the main bus (CPU-Memory) and a secondary bus specific to peripherals (such as PCI Express). The DMA acts as a \u0026ldquo;bridge\u0026rdquo; between the two. Transfers between devices on the secondary bus do not affect the CPU, and sending to the RAM steals the fewest cycles possible. Advantage: the CPU is practically free during the transfer of large blocks. Drawback: requires dedicated hardware (a DMA controller); higher cost and complexity. Comparison table # Characteristic Polling Interrupt DMA HW complexity Minimal Medium High SW complexity Low Medium Medium CPU efficiency Very low Good Very high for large blocks I/O speed Limited Medium High Overhead per transfer High (continuous loop) Medium (context per event) Low (CPU does not take part in each word) Typical applications Simple MCUs Keyboard, mouse, timers Disk, network, audio/video How to choose the right technique Polling: a device with a very fast response (\u0026lt; 1 µs) where the cost of an interrupt would be greater than the wait, or very simple embedded systems without multitasking. Interrupt: most general-purpose devices (keyboard, mouse, UART, timers). DMA: large, frequent transfers (hard disks, SSDs, network cards, audio and video). Calculating the I/O overhead # The I/O overhead measures the fraction of CPU cycles consumed to handle I/O operations:\n$$\\text{Overhead (\\%)} = \\frac{\\text{operations/s} \\times \\text{cycles/operation}}{\\text{total cycles/s}} \\times 100$$Where:\n$$\\text{operations/s} = \\frac{\\text{transfer rate (B/s)}}{\\text{size of the transferred unit (B)}}$$$$\\text{total cycles/s} = f_{\\text{clock}}$$The steps to solve any overhead problem:\n1 Operations per second Calculate how many I/O operations occur per second (rate ÷ unit size). 2 Cycles per second Multiply by the number of CPU cycles spent per operation. 3 Fraction of the clock Divide by the clock frequency. 4 Percentage Multiply by 100 to get the percentage. Example: interrupt overhead of a mouse A USB mouse sends 40 reports per second, each with 4 bytes. The CPU (1 GHz) spends 1000 cycles per interrupt.\nWith interrupts: $$\\text{Overhead} = \\frac{40 \\times 1000}{1 \\times 10^9} = \\frac{40{,}000}{10^9} = 0.004\\%$$Practically negligible: the CPU spends only 4 out of every 100,000 cycles handling the mouse.\nWith polling at 1 kHz (1000 checks/s), 100 cycles per check: $$\\text{Overhead}_{\\text{polling}} = \\frac{1000 \\times 100}{10^9} = 0.01\\%$$Also small for the mouse. But the two numbers come from different premises, and it is worth saying so out loud: the cost per interrupt here is 1000 cycles and the cost per polling check is 100. The difference is real: handling an interrupt requires saving context, jumping to the service routine, and restoring everything afterward, whereas a polling check can be just a register read and a test. In the disk examples further on, the interrupt cost used is 100 cycles, assuming a service routine much leaner than the mouse\u0026rsquo;s.\nBeware of the conclusion \u0026ldquo;polling is always worse\u0026rdquo; Common sense says that polling wastes CPU and interrupts do not, and for a long time that was exactly how it was. But with very fast devices the math flips, and today polling has come back precisely where performance matters most.\nThe reason is that an interrupt has a fixed cost per event: saving context, jumping to the service routine, restoring. When the device takes milliseconds to respond, this cost is irrelevant. When it responds in a few microseconds and delivers hundreds of thousands of operations per second, the cost of interrupts starts to dominate, and busy-waiting for a few microseconds becomes cheaper than being interrupted.\nThat is why an NVMe SSD at several GB/s is not the case where polling would be catastrophic: it is one of the cases where it is the right choice. Linux has NAPI for high-speed networks, which turns off interrupts and starts sweeping the queue under load; io_uring offers a polling mode for disk I/O; and frameworks like DPDK and SPDK use polling in place of interrupts, usually with a core dedicated to sweeping queues. The \u0026ldquo;How to choose the right technique\u0026rdquo; box already pointed in this direction by saying that polling suits devices with a very fast response; here is the reason.\nTeaching note on the following example The calculation below uses a simplified and historically classic model: one interrupt per small block and a fixed transfer size. This is useful for comparing the three I/O techniques, but modern devices (especially NVMe SSDs and high-speed networks) use more sophisticated mechanisms of queuing, interrupt coalescing, and transfer.\nExample: reading 1 MB from disk and comparing the three techniques A system with a 1 GHz CPU reading 1 MB from a disk with 5 ms latency and a 100 MB/s rate.\nWith polling:\nTransfer time: \\(1\\ \\text{MB} / 100\\ \\text{MB/s} = 10\\ \\text{ms}\\) $$\\text{Wasted cycles} = (5 + 10) \\times 10^{-3} \\times 10^9 = 15 \\times 10^6\\text{ cycles}$$ CPU practically blocked during the entire operation.\nWith interrupts (teaching model: 1 interrupt per 512 B sector): $$\\text{Interrupts} = \\frac{1\\ \\text{MB}}{512\\ \\text{B}} = 2048\\text{ interrupts}$$ $$\\text{Overhead} = 2048 \\times 100\\ \\text{cycles} = 204{,}800\\text{ cycles}$$ CPU free for most of the operation, but still needs to handle many interrupts.\nWith DMA:\nThe CPU programs the DMA just once (cost ≈ tens of cycles), the DMA transfers the whole block autonomously, 1 single interrupt at the end. $$\\text{Overhead}_{\\text{DMA}} \\approx \\text{programming} + 100\\text{ cycles (1 IRQ)} \\ll 204{,}800$$ Technique Estimated overhead Polling 15,000,000 cycles (CPU blocked) Interrupt ~204,800 cycles DMA ~tens + 100 cycles Example: identifying techniques from pseudocode Three distinct implementations for transferring data from a device:\nImplementation 1: Configure device → Start read → wait in a loop until done → Store → Process.\nTechnique: polling. The keyword is the busy-wait loop before storing.\nImplementation 2: Configure device → Set up the ISR routine → Keep running; when the IRQ arrives, the ISR stores the data and returns → Process.\nTechnique: interrupt. The keyword is the service routine (ISR) triggered by the device.\nImplementation 3: Configure the DMA controller (addresses, size) → Enable → Controller transfers without CPU intervention → Interrupt at the end → Process.\nTechnique: DMA. The keyword is \u0026ldquo;without CPU intervention\u0026rdquo; during the transfer.\nRecommended scenarios:\nPolling: exact control of the moment of reading without interrupt latencies (critical real-time data acquisition). Interrupt: asynchronous devices of moderate speed (mouse, keyboard, UART). DMA: high volumes of continuous data (video, disk, network). Conclusion and next articles # I/O is the point where the computer meets the physical world, and managing this boundary is fundamentally a problem of speed difference. Polling wastes CPU on busy-waiting; interrupts delegate the notification to the hardware, freeing the CPU for useful work; DMA frees the CPU completely for large blocks. The right choice depends on the volume of data, the frequency of events, and the tolerance for overhead.\nIn the next article, we will explore advanced processor architectures: RISC vs. CISC, performance metrics, Flynn\u0026rsquo;s taxonomy, and how modern processors move toward parallelism at multiple levels.\nSee you then!\n","date":"2026-05-12","externalUrl":null,"permalink":"/2026/05/input-output/","section":"Blog","summary":"Busy-waiting on a 10 ms disk, a 1 GHz CPU wastes ten million cycles. We compare the three ways of handling input and output (polling, interrupts, and DMA), calculate the overhead of each one, and see why polling came back with NVMe SSDs.","title":"Input and Output: How the Computer Talks to the World","type":"posts"},{"content":"","date":"2026-05-12","externalUrl":null,"permalink":"/tags/interrupcoes/","section":"tags","summary":"","title":"interrupções","type":"tags"},{"content":"","date":"2026-05-12","externalUrl":null,"permalink":"/tags/interrupts/","section":"tags","summary":"","title":"interrupts","type":"tags"},{"content":"","date":"2026-05-12","externalUrl":null,"permalink":"/tags/polling/","section":"tags","summary":"","title":"polling","type":"tags"},{"content":"","date":"2026-05-07","externalUrl":null,"permalink":"/tags/ast/","section":"tags","summary":"","title":"AST","type":"tags"},{"content":"","date":"2026-05-07","externalUrl":null,"permalink":"/tags/bytecode/","section":"tags","summary":"","title":"bytecode","type":"tags"},{"content":"","date":"2026-05-07","externalUrl":null,"permalink":"/tags/compiladores/","section":"tags","summary":"","title":"compiladores","type":"tags"},{"content":"","date":"2026-05-07","externalUrl":null,"permalink":"/tags/compilers/","section":"tags","summary":"","title":"compilers","type":"tags"},{"content":"In the previous article, Program Execution, we described a classic compilation pipeline with six stages: lexical analysis, syntax analysis, semantic analysis, intermediate code generation, optimization, and object code generation. Python has something rare from a teaching standpoint: it exposes several important stages of this process through standard library modules. Not as cleanly separated as in a theoretical compiler, but enough to make much of the theory concrete.\nWe will use the same expression from the previous article as our common thread:\nx = y + z * 2 Each section is inspired by one of those stages, looking for a didactic correspondence with tools exposed by CPython.\nThe real scope of this article This article builds a bridge between compiler theory and CPython\u0026rsquo;s introspection tools. Some stages show up very clearly (tokenize, ast, compile, dis); others appear only indirectly or partially. So the goal here is not to claim that Python publicly exposes \u0026ldquo;the six phases\u0026rdquo; of a classic compiler in isolation, but to show how several of these transformations can be observed in practice.\nBy the way, we often hear that \u0026ldquo;Python is an interpreted language\u0026rdquo;, but that is only half the truth. In the standard implementation (CPython), the code is not read and executed line by line straight from the original text. Before running any statement, Python compiles your source code to a lower-level intermediate format called bytecode. Only then is this bytecode interpreted by an internal virtual machine (the Python Virtual Machine, PVM). This silent compilation process is exactly what we are going to dissect now.\nPhase 1: lexical analysis with the tokenize module # The first phase turns the source code into a sequence of tokens. The tokenize module does exactly what a real lexical analyzer does, and lets us inspect the result:\nimport tokenize import io code = \u0026#34;x = y + z * 2\u0026#34; tokens = tokenize.generate_tokens(io.StringIO(code).readline) for tok in tokens: print(tok) Output (simplified):\nTokenInfo(type=1 (NAME), string=\u0026#39;x\u0026#39;, ...) # identifier TokenInfo(type=55 (OP), string=\u0026#39;=\u0026#39;, ...) # operator TokenInfo(type=1 (NAME), string=\u0026#39;y\u0026#39;, ...) TokenInfo(type=55 (OP), string=\u0026#39;+\u0026#39;, ...) TokenInfo(type=1 (NAME), string=\u0026#39;z\u0026#39;, ...) TokenInfo(type=55 (OP), string=\u0026#39;*\u0026#39;, ...) TokenInfo(type=2 (NUMBER), string=\u0026#39;2\u0026#39;, ...) TokenInfo(type=4 (NEWLINE),string=\u0026#39;\\n\u0026#39;, ...) TokenInfo(type=0 (ENDMARKER), ...) Token type numbers change between versions The same warning this article makes about bytecode applies here: token.OP is 55 in Python 3.12, but it was 54 in 3.11 and earlier, because 3.12 added new token types in the middle of the numbering. The other ones in this output have been stable for a long time (NAME = 1, NUMBER = 2, NEWLINE = 4, ENDMARKER = 0), but relying on any of them by number is fragile.\nThe second version of the code, right below, is immune to this: it uses token.tok_name to translate the number into a name, which is the correct way to write code that survives an interpreter upgrade.\nCompare with what we described in the theory in earlier articles:\nInput: x = y + z * 2 Tokens: [ID: x] [OP: =] [ID: y] [OP: +] [ID: z] [OP: *] [NUM: 2] tokenize produces exactly this list, plus position metadata (line and column) that would be used for precise error messages.\n# More readable version: show only type and value import tokenize, io, token code = \u0026#34;x = y + z * 2\u0026#34; for tok in tokenize.generate_tokens(io.StringIO(code).readline): if tok.type not in (token.NEWLINE, token.NL, token.ENDMARKER, token.ENCODING): print(f\u0026#34;{token.tok_name[tok.type]:10} | {tok.string!r}\u0026#34;) NAME | \u0026#39;x\u0026#39; OP | \u0026#39;=\u0026#39; NAME | \u0026#39;y\u0026#39; OP | \u0026#39;+\u0026#39; NAME | \u0026#39;z\u0026#39; OP | \u0026#39;*\u0026#39; NUMBER | \u0026#39;2\u0026#39; Phase 2: syntax analysis with the ast module # The second phase organizes the tokens into a hierarchical structure that reflects the grammar of the language: the Abstract Syntax Tree (AST). The ast module exposes this structure directly:\nimport ast code = \u0026#34;x = y + z * 2\u0026#34; tree = ast.parse(code) print(ast.dump(tree, indent=2)) Output (simplified):\nModule( body=[ Assign( targets=[Name(id=\u0026#39;x\u0026#39;)], value=BinOp( left=Name(id=\u0026#39;y\u0026#39;), op=Add(), right=BinOp( left=Name(id=\u0026#39;z\u0026#39;), op=Mult(), right=Constant(value=2) ) ) ) ] ) Compare with the tree drawn in the previous article, here in text form:\nassign / \\ x + / \\ y * / \\ z 2 It is the same structure: ast.dump is just the serialized version of the tree. The BinOp with Mult appears deeper than the BinOp with Add, capturing the precedence of * over +.\nTransforming the AST before compilation # The ast module lets us inspect and also transform the tree before compiling, the basis of tools like linters, formatters, and optimizers. We can, for example, write a script that walks the tree looking for multiplications by powers of 2 and replaces them with bit shifts. As we saw in the article on The Hypothetical Processor, the ALU performs shifts much faster than it does with complex adders:\nimport ast class ReplaceMultiplicationWithShift(ast.NodeTransformer): \u0026#34;\u0026#34;\u0026#34; Replaces multiplications by constants that are powers of 2 with left shifts (more efficient). \u0026#34;\u0026#34;\u0026#34; def visit_BinOp(self, node): self.generic_visit(node) # process children first if isinstance(node.op, ast.Mult) and isinstance(node.right, ast.Constant): n = node.right.value if isinstance(n, int) and n \u0026gt; 0 and (n \u0026amp; (n - 1)) == 0: shift = n.bit_length() - 1 print(f\u0026#34;Optimizing: * {n} → \u0026lt;\u0026lt; {shift}\u0026#34;) return ast.BinOp( left=node.left, op=ast.LShift(), right=ast.Constant(value=shift), ) return node code = \u0026#34;result = z * 4\u0026#34; tree = ast.parse(code) optimized_tree = ReplaceMultiplicationWithShift().visit(tree) ast.fix_missing_locations(optimized_tree) compiled_code = compile(optimized_tree, \u0026#34;\u0026lt;transformed\u0026gt;\u0026#34;, \u0026#34;exec\u0026#34;) print(ast.dump(optimized_tree, indent=2)) # Proof that the transformation reached the bytecode: import dis dis.dis(compiled_code) In the dis output there is a BINARY_OP 3 (\u0026lt;\u0026lt;) where there would otherwise be BINARY_OP 5 (*): the rewritten tree became different bytecode, and that is the only evidence that matters. Without this step, the example would show only that the AST changed, not that the change survived compilation.\nWhat this example really shows This transformation with ast.NodeTransformer is a great example of syntactic rewriting of the AST. It looks like a compiler optimization, but it should not be sold here as a \u0026ldquo;guaranteed real performance optimization\u0026rdquo; in Python. In CPython, the execution cost involves Python objects, dynamic dispatch, and other factors that go far beyond the abstract arithmetic operation.\nAnd there is a more serious problem: the transformation is wrong Not wrong as code: wrong as an optimization. The rewrite z * 4 → z \u0026lt;\u0026lt; 2 only holds if z is an integer. If z is a float, z * 4 works perfectly and z \u0026lt;\u0026lt; 2 raises TypeError: unsupported operand type(s) for \u0026lt;\u0026lt;: 'float' and 'int'. The transformer only looks at syntax: it sees a multiplication by a power of 2 and rewrites it, knowing nothing about the type of z.\nAnd that is exactly the argument for why a real compiler needs semantic analysis before optimizing: it is what discovers the types, and without types there is no way to know whether a rewrite preserves the meaning of the program. The next section is precisely about that phase. An optimizer that runs without it does not optimize: it gambles.\nPhase 3: semantics and runtime checks # In traditional compilers, semantic analysis includes tasks such as type checking, name usage, operation compatibility, and scope consistency. In CPython, an important part of these checks happens at runtime, and not as a public phase separate from compilation.\nSo this stage should be understood here only approximately: Python does not expose a public \u0026ldquo;complete semantic analysis\u0026rdquo; API comparable to that of a classic static compiler.\nA useful example is to show that certain expressions are accepted syntactically but fail when executed:\n# Type check at runtime (not at compile time): try: result = 3 + \u0026#34;5\u0026#34; except TypeError as e: print(f\u0026#34;TypeError at runtime: {e}\u0026#34;) # TypeError at runtime: unsupported operand type(s) for +: \u0026#39;int\u0026#39; and \u0026#39;str\u0026#39; It is also worth noting that ast.literal_eval() is not a general semantic analyzer for Python. It only evaluates, without executing code, simple literal structures such as numbers, strings, lists, tuples, dictionaries, sets, None, and booleans. The documentation advises against using it with untrusted data, because a small input can exhaust the interpreter\u0026rsquo;s memory or stack.\nimport ast # ast.literal_eval evaluates literals without executing arbitrary code # (but it is not recommended for untrusted strings: it can exhaust memory or stack) try: result = ast.literal_eval(\u0026#34;[1, 2, 3]\u0026#34;) print(result) # [1, 2, 3] print(type(result)) # \u0026lt;class \u0026#39;list\u0026#39;\u0026gt; # Trying to evaluate a non-literal expression → ValueError ast.literal_eval(\u0026#34;x + 1\u0026#34;) # ERROR: contains a variable except ValueError as e: print(f\u0026#34;Expected failure: {e}\u0026#34;) The correct reading of this phase, therefore, is:\nin classic compilers: explicit semantic analysis; in CPython: several relevant semantic checks show up at runtime. Phase 4: compile() and object code generation # The compile() function converts Python source code (or an already built AST) into a code object. This object can be executed by exec() or eval().\nThis should not be interpreted as if Python exposed a classic intermediate representation in the style of TAC (Three-Address Code). The object generated by compile() is already much closer to the executable form used by the CPython virtual machine than to a didactic compiler IR.\n# compile(source, filename, mode) # mode: \u0026#39;exec\u0026#39; for modules, \u0026#39;eval\u0026#39; for expressions, \u0026#39;single\u0026#39; for the REPL obj = compile(\u0026#34;x = y + z * 2\u0026#34;, \u0026#34;\u0026lt;string\u0026gt;\u0026#34;, \u0026#34;exec\u0026#34;) print(type(obj)) # \u0026lt;class \u0026#39;code\u0026#39;\u0026gt; print(obj.co_consts) # constants used: (2, None) print(obj.co_names) # global variable names: (\u0026#39;y\u0026#39;, \u0026#39;z\u0026#39;, \u0026#39;x\u0026#39;) print(obj.co_stacksize) # maximum stack size needed The code object is CPython\u0026rsquo;s internal executable unit, and it is this kind of object that can be serialized into .pyc files for load caching.\nAbout writing .pyc by hand It is technically possible to serialize a code object with the marshal module and assemble a .pyc manually, but it is not worth it, not even as a demonstration. The marshal format is specific to Python, deliberately not documented in detail, and changes between versions; and the .pyc header starts with a magic number that identifies the exact interpreter version: making up this number produces a file that Python refuses to import.\nTo observe .pyc in practice, the way is to let Python itself generate it, which is what the next section does.\nPractical example: letting Python generate the compiled cache # Instead of writing a .pyc file by hand, the proper way is to let Python generate the cache automatically.\n# Save this content in a file called example_module.py: # # value = 42 # def double(x): # return x * 2 # # Then, in another Python file or in the REPL: import example_module print(example_module.value) # 42 print(example_module.double(10)) # 20 After the import, Python tends to create a __pycache__/ directory with a .pyc file corresponding to the module, when appropriate. This file stores a version compiled to bytecode to speed up future loads.\nPhase 5: some observable compiler optimizations # CPython performs some optimizations during compilation, but they do not appear as a separate, configurable \u0026ldquo;public phase\u0026rdquo; the way they do in a classic compiler. The easiest example to observe here is constant folding: expressions made up only of constants can be precomputed at compile time.\nimport dis # Without constant folding: variables must be resolved at runtime code_vars = compile(\u0026#34;result = y + z * 2\u0026#34;, \u0026#34;\u0026lt;string\u0026gt;\u0026#34;, \u0026#34;exec\u0026#34;) # With constant folding: pure constants are evaluated at compile time code_const = compile(\u0026#34;result = 3 + 4 * 2\u0026#34;, \u0026#34;\u0026lt;string\u0026gt;\u0026#34;, \u0026#34;exec\u0026#34;) print(\u0026#34;With variables — constants in the AST:\u0026#34;) print(compile(\u0026#34;y + z * 2\u0026#34;, \u0026#34;\u0026lt;string\u0026gt;\u0026#34;, \u0026#34;eval\u0026#34;).co_consts) # (2,) ← only the literal 2 is a constant print(\u0026#34;\\nWith only constants — after constant folding:\u0026#34;) print(compile(\u0026#34;3 + 4 * 2\u0026#34;, \u0026#34;\u0026lt;string\u0026gt;\u0026#34;, \u0026#34;eval\u0026#34;).co_consts) # (11,) ← Python precomputed 3 + 4*2 = 11 at compile time! Why None shows up in exec and not in eval Compare with the previous section, where compile(\u0026quot;x = y + z * 2\u0026quot;, \u0026quot;\u0026lt;string\u0026gt;\u0026quot;, \u0026quot;exec\u0026quot;).co_consts returned (2, None). The difference is the mode. In exec mode, the code object is a module, and every module implicitly ends with return None, hence the extra constant. In eval mode, the code object is an expression, which returns its own value: there is no implicit return, and None does not appear.\nIt is worth knowing this because the order and contents of co_consts have already changed between CPython versions, and this is the kind of detail where an example copied from an old text stops matching what the interpreter shows. The values here were checked on Python 3.12.\nPhase 6: object code with the dis module # The dis (disassembler) module shows the bytecode instructions that CPython is going to execute, the closest we can get, in pure Python, to inspecting the code that the CPython virtual machine will run.\nImportant: bytecode depends on the CPython version The dis module works on CPython\u0026rsquo;s bytecode, and this bytecode is an implementation detail. It can change between Python versions and should not be treated as a stable language that is portable across VMs or releases.\nimport dis def calculate(y, z): x = y + z * 2 return x dis.dis(calculate) Typical output (Python 3.11 and 3.12; from 3.13 on, the two LOAD_FAST become a single LOAD_FAST_LOAD_FAST instruction):\n2 0 RESUME 0 3 2 LOAD_FAST 0 (y) 4 LOAD_FAST 1 (z) 6 LOAD_CONST 1 (2) 8 BINARY_OP 5 (*) 12 BINARY_OP 0 (+) 16 STORE_FAST 2 (x) 4 18 LOAD_FAST 2 (x) 20 RETURN_VALUE Each line is a bytecode instruction: an opcode (such as LOAD_FAST, BINARY_OP) plus an argument. Notice the correspondence with the TAC from the previous article:\nCPython bytecode Approximate logical role LOAD_FAST y loads y onto the stack LOAD_FAST z loads z onto the stack LOAD_CONST 2 loads the constant 2 BINARY_OP * combines z and 2 BINARY_OP + combines y with the previous result STORE_FAST x stores the result in x This table does not aim to reproduce a classic TAC literally; it only shows the approximate logical correspondence between CPython\u0026rsquo;s stack bytecode and the operations of the original expression.\nThe order may look inverted, but CPython uses an evaluation stack: each LOAD pushes a value; each BINARY_OP pops two operands, operates, and pushes the result.\nStack architecture vs. registers If you followed our articles on the CPU, you should remember that our hypothetical processor used an accumulator or registers to do arithmetic. The CPython interpreter, however, simulates a Stack Machine. In it, we do not say where to save the result of an addition in the instruction itself; operations simply take the data at the top of the stack, compute, and put the result back on top.\nInspecting the bytecode of a complete module # import dis, ast # Complete pipeline: source code → tokens → AST → bytecode source_code = \u0026#34;\u0026#34;\u0026#34; def sum_of_squares(a, b): return a**2 + b**2 \u0026#34;\u0026#34;\u0026#34; print(\u0026#34;=== TOKENS ===\u0026#34;) import tokenize, io, token as tok_mod for t in tokenize.generate_tokens(io.StringIO(source_code).readline): if t.type not in (tok_mod.NL, tok_mod.NEWLINE, tok_mod.ENDMARKER, tok_mod.ENCODING, tok_mod.INDENT, tok_mod.DEDENT): print(f\u0026#34; {tok_mod.tok_name[t.type]:10} {t.string!r}\u0026#34;) print(\u0026#34;\\n=== AST ===\u0026#34;) tree = ast.parse(source_code) print(ast.dump(tree, indent=2)) print(\u0026#34;\\n=== BYTECODE ===\u0026#34;) compiled_code = compile(source_code, \u0026#34;\u0026lt;inline\u0026gt;\u0026#34;, \u0026#34;exec\u0026#34;) # the function is inside the module — we need to extract it exec(compiled_code, globals()) dis.dis(sum_of_squares) Putting the pieces together # Before the table, the whole expression at once. The figure below puts side by side the four forms that x = y + z * 2 took throughout the article, with the tool that leads from one to the next. The colors follow each token from start to finish, and the numbers mark the same evaluation order in the tree and in the bytecode: the stack executes the operators from the bottom to the top of the AST. The tree has the same drawing as the figure in Program Execution.\nSource code, tokens, AST, and bytecode of the same expression. Spaces vanish in tokenization, precedence becomes shape in the tree, and the bytecode executes the operators from the bottom to the top of the AST. The following table maps each theoretical phase described in Program Execution to the corresponding Python tool:\nPhase Theory Python tool What to observe 1. Lexical analysis Tokenization tokenize List of tokens with type and value 2. Syntax analysis Parser / AST ast.parse() + ast.dump() Hierarchical tree with precedences 3. Semantics / checks Types and operational consistency Runtime (TypeError, etc.) An important part of the consistency is checked during execution 4. Executable code object Internal executable representation compile() → co_consts, co_names, co_stacksize Code object that CPython can execute 5. Optimization Constant folding, etc. co_consts (compare with/without constants) Constant expressions precomputed 6. Object code Bytecode dis.dis() Evaluation stack bytecode instructions Reading the table correctly The table above does not mean that CPython exposes a public, independent API for each phase of a classic compiler. Rather, it shows an approximate didactic correspondence between the theory and the tools that Python offers for introspection.\nConclusion and next articles # Python is a good language for observing several internal transformations of the compilation and execution process. tokenize, ast, compile, and dis make concrete stages that, in many compilers, stay hidden behind internal tools or outputs that are hard to inspect.\nThe next time you run python script.py, you will have a much clearer view of what happens behind the scenes.\nFor those who want to go down one more level, the CPython InternalDocs, maintained alongside the source code and pointed to by the developer guide, document the compiler from the inside: the same phases seen here, but from the point of view of whoever maintains the interpreter, with the source files for each stage. It is free, official, and covers the pipeline of this article.\nAbout __pycache__ Python stores compiled .pyc versions in __pycache__ to speed up module loading. This process is automatic: the interpreter compares the cache with the source and recompiles when necessary.\nIn the next article, we will talk about input and output, that is, how the computer communicates with the outside world: polling, interrupts, and direct memory access (DMA), and why the choice of I/O mechanism affects the performance of the whole system.\nSee you then!\n","date":"2026-05-07","externalUrl":null,"permalink":"/2026/05/python-bytecode-compilers/","section":"Blog","summary":"Python does not run your code line by line: first, CPython compiles it to bytecode. With the tokenize, ast, compile, and dis modules, we follow a simple expression from tokens to syntax tree to bytecode, right in the interpreter.","title":"Python and Compilers: A Didactic Bridge to the CPython Pipeline","type":"posts"},{"content":"","date":"2026-05-05","externalUrl":null,"permalink":"/tags/assembly/","section":"tags","summary":"","title":"Assembly","type":"tags"},{"content":"","date":"2026-05-05","externalUrl":null,"permalink":"/tags/java/","section":"tags","summary":"","title":"Java","type":"tags"},{"content":" In the previous article, Instruction Representation, we saw how instructions are encoded in bits and how addressing modes determine where the processor finds each operand. But you write x = y + z in Python, not 0001 10110100. Someone has to do the translation. That someone is the toolchain that turns source code into machine code: the assembler, the compiler, the linker, and the loader.\nUnderstanding this chain means understanding why C and C++ usually produce native executables, why Java distributes bytecode for the JVM, and why, when you run python script.py, the CPython interpreter first compiles the code to bytecode and then executes it in its virtual machine. In every case, the final goal is the same: to arrive at instructions the hardware can execute, directly or indirectly.\nFrom human language to silicon: a brief history # The first programming languages were created in response to a simple demand: to make programming less error-prone and closer to human thought. The following table shows the timeline of some languages that shaped software development:\nYear Language Created by Characteristics 1957 FORTRAN John Backus (IBM) Scientific/engineering: one of the first high-level compilers 1958 LISP John McCarthy (MIT) List processing; basis for AI 1959 COBOL CODASYL committee (with Grace Hopper as consultant) Business/commercial: readable like English prose 1960 ALGOL 60 International committee Structured algorithms; influenced C and Pascal 1964 BASIC John Kemeny and Thomas Kurtz (Dartmouth) Simple, educational 1970 Pascal Niklaus Wirth (ETH Zürich) Structured, educational 1972 C Dennis Ritchie (Bell Labs) Operating systems; basis of Unix 1983 Ada Jean Ichbiah and team (CII Honeywell Bull) Military, real time 1983 C++ Bjarne Stroustrup (Bell Labs) Object-oriented; extension of C 1987 Perl Larry Wall Text processing, scripts 1991 Python Guido van Rossum (CWI) General purpose, readable, multiparadigm 1995 JavaScript Brendan Eich (Netscape) Web; dynamic, front end and back end (Node.js); ubiquitous in browsers 1995 PHP Rasmus Lerdorf Server-side web; basis of WordPress and much of the web 1995 Ruby Yukihiro Matsumoto Pure OO, elegant; popularized by the Rails framework 1996 Java James Gosling (Sun) Portable, object-oriented; \u0026ldquo;Write once, run anywhere\u0026rdquo; 2000 C# Anders Hejlsberg (Microsoft) Microsoft/.NET; OO, similar to Java; Windows/Azure ecosystem 2003 Scala Martin Odersky (EPFL) JVM; combines OO and functional programming 2009 Go Griesemer, Pike, and Thompson (Google) Native concurrency, compiled, modern servers 2012 TypeScript Anders Hejlsberg (Microsoft) Typed superset of JavaScript; large-scale web projects 2014 Swift Chris Lattner (Apple) Replaces Objective-C for iOS/macOS 2015 Rust Graydon Hoare (Mozilla) Memory safety without a garbage collector; high-performance systems 2016 Kotlin JetBrains Modern, concise; official Android language A few dates deserve a note, because different versions circulate out there. COBOL sometimes appears as 1958, but the CODASYL committee was only formed in May 1959 and the specification was approved in January 1960. The year 1958 belongs to ALGOL 58, a different language. And Ada appears as 1979, the year of the preliminary manual, as 1980, the year of MIL-STD-1815, and as 1983, the year of the MIL-STD-1815A and ANSI standardization, which is the canonical date.\nMany others could be listed, but these are some of the most influential and representative. Each of them was designed with specific goals in mind (readability, performance, portability, safety, or ease of use) and each contributed to the evolution of the software development ecosystem.\nThe abstraction hierarchy # Languages form an abstraction pyramid: the higher the level, the closer to human language, and the farther from the hardware. Each level must be translated to the level below before the processor can execute the code.\nLevel Type Example How it is translated Lowest Machine language (binary) 0001 1011 0100 Directly executable by the processor Low Assembly (mnemonics) LDA 180 Assembler → machine language High High-level language x = y + z Compiler or interpreter The only language the processor understands The processor executes only machine language: sequences of bits. Every higher-level language must be translated, directly or indirectly, into binary code before or during execution. This whole section describes how that translation happens.\nBut how does this translation chain work in practice? Let\u0026rsquo;s descend to the level closest to the hardware and start with the most direct process: assembling Assembly code.\nThe assembly process (Assembly) # The assembler converts assembly code into machine language. The translation is practically 1:1: each mnemonic corresponds to one machine instruction. The assembler works in two passes:\nPass What it does 1st pass Scans the source code and builds the symbol table, which maps labels to addresses 2nd pass Generates the machine code, replacing each reference with the address from the table Why two passes? When the assembler finds a label being used before it is defined (a forward reference), it does not know the address yet. The 1st pass solves this: it goes through all the code, assigns addresses to all the labels, and records them in the symbol table. In the 2nd pass, all references can be resolved without ambiguity.\nAssembly example: an assembly program to compute:\n$$ X = Y + Z - T $$Using the hypothetical processor presented earlier in this series, with:\n\\(Y\\) at address 1F, containing 051 \\(Z\\) at address 20, containing 03E \\(T\\) at address 21, containing 003 \\(X\\) at address 22 The program starts at address 18:\nAddress Assembly Generated code 18 LDA 1F 11F 19 ADD 20 320 1A SUB 21 421 1B STR 22 222 1C HLT 000 1D and 1E (free) (none) 1F Data: \\(Y\\) 051 20 Data: \\(Z\\) 03E 21 Data: \\(T\\) 003 22 Result \\(X\\) written at run time Addresses 1D and 1E are left empty on purpose: the code ends at 1C and the data starts at 1F, and the gap is just the choice of where to begin the data block. In a real program, the assembler decides this layout.\nThe sequence does exactly what is expected:\nloads \\(Y\\) into the accumulator; adds \\(Z\\); subtracts \\(T\\); stores the result in \\(X\\); ends the execution. It is worth checking the arithmetic, since that is what proves the assembly worked. In decimal, \\(Y = \\text{051}_{16} = 81\\), \\(Z = \\text{03E}_{16} = 62\\), and \\(T = \\text{003}_{16} = 3\\), so \\(X = 81 + 62 - 3 = 140\\). At the end of the execution, address 22 contains \\(140_{10} = \\text{08C}_{16}\\).\nThe execution cycle for the LDA 1F instruction (which we already know from the article on the hypothetical processor) works exactly like this:\nStep Operation Description 1 \\(MAR \\leftarrow PC\\) The instruction address goes to the MAR 2 \\(MDR \\leftarrow M[MAR]\\) Fetches the LDA 1F instruction from memory 3 \\(IR \\leftarrow MDR;\\ PC \\leftarrow PC + 1\\) Instruction loaded into the IR; PC advances 4 Decode: LDA The CU identifies the operation and extracts address 1F 5 \\(MAR \\leftarrow 1F\\) The operand address goes to the MAR 6 \\(MDR \\leftarrow M[1F]\\) Fetches the operand value from memory 7 \\(ACC \\leftarrow MDR\\) The accumulator receives the value Compilation: a classic transformation pipeline # While the assembler translates instructions very close to the hardware, the compiler solves a broader problem: transforming high-level code into a form executable on the target architecture. A classic compilation pipeline goes through stages such as lexical, syntactic, and semantic analysis, generation of an intermediate representation, optimization, and object code generation.\nThis flow is extremely useful as a teaching model, although real compilers may reorder, merge, or even omit some of these stages internally.\n\u0026#x2197; %%{init: {'flowchart': {'useMaxWidth': false}}}%% flowchart LR A([Source Code]) --\u003e B[\"1. Lexical Analysis\n(Tokenization)\"] B --\u003e C[\"2. Syntax Analysis\n(Parser / AST)\"] C --\u003e D[\"3. Semantic Analysis\n(Types and Scope)\"] D --\u003e E[\"4. Intermediate\nCode (TAC)\"] E --\u003e F[\"5. Optimization\"] F --\u003e G[\"6. Object Code\"] G --\u003e H([Linker]) H --\u003e I([Executable]) Phase 1: lexical analysis (tokenization) # The lexical analyzer reads the source code character by character and groups the characters into tokens, the smallest units of meaning. It discards comments and whitespace.\nInput: x = y + z * 2 Tokens: [ID: x] [OP: =] [ID: y] [OP: +] [ID: z] [OP: *] [NUM: 2] Each token has a type (identifier, operator, numeric literal, keyword) and a value. Tokenization errors are the first to be detected.\nPhase 2: syntax analysis (parser) # The parser checks whether the sequence of tokens obeys the grammar of the language. If so, it builds the Abstract Syntax Tree (AST), which represents the hierarchical structure of the code.\nThe figure below shows the AST of x = y + z * 2. Each node sits in the column of its token, and so the tree holds two readings: from left to right, it gives back the source code; from bottom to top, it gives the order in which the operations happen.\nThe AST of x = y + z * 2, with the colors of the token categories from lexical analysis. The multiplication, being deeper, is evaluated first; the assignment, at the root, last. The AST captures operator precedence: * is deeper in the tree than +, which means it will be executed first.\nPhase 3: semantic analysis # Checks the meaning of the code beyond its structure:\nWere all variables declared before use? Are the types compatible in operations and assignments? Is the number of arguments in function calls correct? int x; x = \u0026#34;hello\u0026#34;; // Semantic ERROR: correct syntax, but incompatible type Semantic errors go unnoticed by the parser (the syntax x = something is valid), but the semantic analyzer detects them by checking type compatibility.\nPhase 4: intermediate code generation # Produces a machine-independent representation, which eases portability and later optimization. The most common format is Three-Address Code (TAC): each instruction has at most three operands.\nTAC for x = y + z * 2:\nT1 = z * 2 T2 = y + T1 x = T2 TAC resembles the language of our hypothetical processor: simple instructions, temporary registers, no nested operations.\nPhase 5: optimization # Improves the intermediate code without changing the observable result. Some classic techniques:\nTechnique Effect Dead code elimination Removes code that will never be executed Constant propagation Replaces variables with values known at compile time Common subexpression elimination Avoids recomputing identical expressions Loop unrolling Replicates the loop body to reduce control overhead In practice, real compilers (such as the famous GCC, widely used for languages like C and C++) let the programmer choose how aggressive this optimization is, and these levels have names:\nOption What it does -O0 No optimization. Easy-to-debug code: each source line corresponds to a recognizable chunk of assembly -O1 Basic optimizations, without a large compile-time cost -O2 The level used in most production builds -O3 More aggressive: extra loop transformations (interchange, peeling, unswitching) and vectorization with a more liberal cost model -Os Optimizes for code size, not for speed Watching optimization happen # None of this has to stay abstract. Consider this C function, which is the same expression we have been following since lexical analysis:\nint calculate(int y, int z) { int x = y + z * 2; return x; } Compiling with gcc -S -O0 -masm=intel, the compiler generates twelve instructions, with each variable going back and forth to memory, because without optimization it translates literally what the program said:\ncalculate: push rbp mov rbp, rsp mov DWORD PTR [rbp-20], edi ; stores y on the stack mov DWORD PTR [rbp-24], esi ; stores z on the stack mov eax, DWORD PTR [rbp-24] ; reads z back lea edx, [rax+rax] ; edx = z * 2 mov eax, DWORD PTR [rbp-20] ; reads y back add eax, edx ; eax = y + z*2 mov DWORD PTR [rbp-4], eax ; stores x on the stack mov eax, DWORD PTR [rbp-4] ; reads x back to return it pop rbp ret Now the same file with gcc -S -O2 -masm=intel:\ncalculate: lea eax, [rdi+rsi*2] ret One computing instruction, plus the ret. The compiler realized that the variable x does not need to exist, that the parameters are already in registers (rdi and rsi, by the calling convention), and that x86 has an addressing mode (base plus scaled index, which the previous article described) capable of computing \\(y + z \\times 2\\) in a single lea operation. The twelve instructions of -O0 and the two of -O2 do exactly the same thing.\nThis is why reading optimized assembly is hard: the code no longer has a line-by-line correspondence with the source. And this is why you debug with -O0 and ship with -O2.\nPhase 6: object code generation # Translates the optimized intermediate code into machine instructions for the target processor. Produces an object file (.o on Linux/macOS, .obj on Windows).\nAt this stage, we have a binary file that the CPU understands, but it is still a loose piece. The file contains unresolved external references: for example, if your program uses a function from another file of yours, or functions from the system\u0026rsquo;s standard library (such as a print to write to the screen), the object code points to a \u0026ldquo;void\u0026rdquo;. It needs an integrator. Enter the Linker.\nLinking (Linking) # The linker combines multiple object files and libraries into a single executable, resolving all external references.\nType Description Advantage Disadvantage Static Copies the library code into the executable Self-contained executable Larger file; updating the library requires recompiling Dynamic References .dll (Windows) or .so (Linux), loaded at run time Smaller file; libraries shared among programs Run-time dependency Dynamic linking in practice When the system updates a shared library (for example, libc.so.6 on Linux), all the programs that use it through dynamic linking benefit immediately, without needing to be recompiled. With static linking, each program would have to be recompiled and redistributed individually. On the other hand, dynamic linking creates a run-time dependency: if the library is not installed or is incompatible, the program fails to start, the famous \u0026ldquo;DLL Hell\u0026rdquo; on Windows.\nLinking and portability Dynamic linking reduces the size of executables and eases updates of shared libraries, but it also introduces external dependencies at run time. Static linking, in turn, tends to produce more self-contained executables, although larger and less flexible for maintenance.\nLoading (Loading) # The loader transfers the executable from storage to Main Memory and prepares the environment so that it can start running. In real systems, this may involve not only copying the program into memory, but also adjusting addresses, mapping segments, and preparing shared libraries.\nType Description Absolute loading Fixed addresses: the program is always loaded at the same memory address (inflexible) Relocatable loading Addresses adjusted at load time according to the position in MM, and it uses the base + displacement mode we saw in the previous article Dynamic loading Parts of the program or libraries are loaded on demand at run time With the program properly assembled, compiled, linked, and loaded into Main Memory, the preparatory cycle ends. From this exact moment, the silicon takes over and the CPU starts the perpetual loop of fetch, decode, and execute that we dissected in the previous articles of the series.\nCompilation × interpretation # The distinction between compiled and interpreted languages is useful, but it must be seen as a teaching model, not as an absolute boundary.\nCharacteristic Traditional compilation Traditional interpretation Main translation Before execution During execution Generated artifact Object code, executable, or bytecode Usually there is no prior native executable of the user\u0026rsquo;s program Performance Tends to be higher when there is native code Tends to have higher runtime overhead Portability Depends on the compiled target Depends on the interpreter or virtual machine Typical examples C, C++, Rust, Go CPython, Ruby, shells In practice, modern systems often mix strategies:\ncompilation to bytecode; interpretation by a virtual machine; JIT compilation at run time; caching of intermediate artifacts. That is why it is better to think of a spectrum of translation and execution, not a rigid dichotomy.\nExample: Python, Java, and bytecode Python (CPython) compiles modules to bytecode, which can be cached in .pyc files inside __pycache__. After that, this bytecode is executed by the CPython virtual machine.\nJava, in the traditional flow, uses javac to compile .java files into .class files, which contain JVM bytecode. This bytecode is then executed by the Java Virtual Machine.\nThe main difference is not \u0026ldquo;Python interprets source and Java does not\u0026rdquo;, but the way each ecosystem organizes the stages:\nin CPython, compilation to bytecode and execution usually happen transparently to the user; in Java, the traditional flow clearly separates the generation of the .class from the execution on the JVM. Moreover, modern versions of Java also offer source-file mode for simple single-file programs, which shows that these boundaries are not entirely rigid.\nWhat happens when you run python script.py The python executable was previously compiled, linked, and installed on your system. When you run python script.py, what happens to your program is another stage: CPython reads the source code, compiles it to bytecode, and executes that bytecode in its virtual machine, and it may also reuse caches in __pycache__ when appropriate.\nTyping: when types are checked # The discussion about compilation and interpretation connects directly with typing: how and when a language checks type compatibility between variables and operations.\nStatic × dynamic # Type When types are checked Examples Static At compile time C, Java, Rust, Go Dynamic At run time Python, JavaScript, Ruby Strong × weak # Type Behavior Example with 3 + \u0026quot;5\u0026quot; Strong Forbids operations between incompatible types without explicit conversion Error (TypeError in Python) Weak Allows implicit type coercion \u0026quot;35\u0026quot; (JavaScript) By language # Language Static/Dynamic Note on type strength C Static Allows several implicit coercions and potentially dangerous conversions Java Static More restrictive in operations between incompatible types Python Dynamic Rejects incompatible operations without explicit conversion (3 + \u0026quot;5\u0026quot; → TypeError) JavaScript Dynamic Frequently performs implicit coercions Typing and execution strategy There is a frequent correlation between static typing and compilation, and between dynamic typing and execution by interpreters or virtual machines. But this is not an absolute rule.\nJava is statically typed, but runs on the JVM. Python is dynamically typed, but still compiles to bytecode. In other words: typing and translation/execution strategy are related but independent dimensions.\nConclusion and next articles # The path from x = y + z to the bits the processor executes involves several transformations: source code analysis, generation of internal representations, production of object code, linking, loading, and execution. Depending on the language and the environment, some of these stages are explicit to the programmer; others happen invisibly inside the runtime.\nIf you want to keep exploring, three references cover well what this article only introduces. Compiler Explorer shows the assembly generated for any snippet of code, in dozens of languages and architectures, without installing anything. It is this section\u0026rsquo;s gcc -S, with an interface. Crafting Interpreters, by Bob Nystrom, is a complete, free book that builds two implementations of a language from scratch, covering the phases discussed here and going beyond them. And Linkers and Loaders, by John Levine, is the classic reference on the stage that almost no material covers in depth: precisely linking and loading.\nIn the next article, we will explore how the process happens in practice using Python, observing each phase directly in the interpreter. We will see how the tokenize module handles lexical analysis, how the ast module builds the abstract syntax tree, how the compile function generates bytecode, and how the dis module lets us inspect the generated bytecode. It will be an opportunity to see each stage in action, using a high-level language that many of you already know.\nSee you then!\n","date":"2026-05-05","externalUrl":null,"permalink":"/2026/05/program-execution/","section":"Blog","summary":"You write a line of code, and the processor only understands bits. We follow the translation: the two-pass assembler, the compiler phases, linking and loading. Along the way, GCC cuts twelve instructions down to two with -O2, and we see where Python and Java sit between compiling and interpreting.","title":"Program Execution: From Source Code to Machine Code","type":"posts"},{"content":" In the previous article, Data Representation, we saw how the computer encodes characters, integers, and floating-point numbers as bit patterns. But a program is not just data: it is a sequence of instructions that tell the processor what to do with that data. How are these instructions encoded? Where does the processor find the operands they need?\nThese questions are answered by the ISA (Instruction Set Architecture) and by the addressing modes it defines. The ISA is the meeting point between hardware and software: everything a program does must be expressed in terms the processor understands.\nThe ISA: the contract between hardware and software # The ISA (Instruction Set Architecture) is the interface between hardware and software. It defines the set of instructions the processor understands and the model visible to compilers, assemblers, operating systems, and programs.\nThe ISA specifies, among other things:\nthe instruction format; the supported data types; the addressing modes; the registers and their roles; the set of operations the CPU executes natively. It is useful to think of the ISA as a compatibility contract: programs translated for an ISA only run directly on processors that implement that ISA, except through emulation, binary translation, or virtual machines.\nISA is not the same thing as microarchitecture Two processors can implement the same ISA and still be very different internally. This is the case with different x86-64 CPUs from Intel and AMD: the software sees the same ISA, but the internal hardware can vary a lot.\nProgramming languages and ISA Not every high-level language turns directly into instructions of the machine\u0026rsquo;s ISA.\nIn natively compiled languages, such as C and C++, the compiler usually generates machine code for the target ISA. In languages or platforms such as Java and Python, there may be intermediate steps, such as bytecode, a virtual machine, or JIT compilation, before the final execution on the hardware. Even so, ultimately, the actual execution always ends in instructions of the ISA implemented by the processor.\nISAs in the market today The x86-64 family remains dominant on desktops and many servers. The ARM family now goes far beyond smartphones, also appearing in embedded systems, laptops, the cloud, and data centers. RISC-V, in turn, has gained relevance for being an open ISA, especially in research, teaching, and embedded systems.\nA useful classification of instructions # A didactic way to organize machine instructions is to group them by function:\n# Category Examples of mnemonics 1 Data movement LOAD, STORE, MOV 2 Integer arithmetic ADD, SUB, MPY, DIV 3 Floating-point arithmetic FADD, FMPY, FDIV 4 Logic AND, OR, NOT, XOR 5 Shifts SHL, SHR, SAR 6 Control flow JMP, JZ, CALL, RET 7 Input/Output IN, OUT This classification is useful for study, but it should not be treated as a universal taxonomy. Modern real ISAs also include, for example, vector, atomic, cryptographic, system, and synchronization instructions.\nRISC architectures tend to have few instructions per category, each simple and regular. CISC architectures can have dozens of variants per category, including complex operations in a single instruction.\nBut regardless of the category or the complexity, the CPU needs to know how to read that order. Every instruction has to be physically packed into bits, and that is where the design of instruction formats comes in.\nInstruction formats: how many operands? # One of the fundamental decisions in designing an ISA is how many operands each instruction carries. There is a clear trade-off: more operands allow more expressive instructions (and therefore shorter programs), but each instruction takes up more space in memory.\nIn the past, the concern with space was so extreme that designs such as the SEAC (1950) used up to 4 operands: three for the operation and a fourth just to indicate the address of the next instruction. With the popularization of the PC register (Program Counter), which automatically points to the next task, we save those precious bits, allowing leaner formats.\nTo illustrate, let us compute the expression:\n$$X = A \\times (B + C \\times D - E / F)$$in three different formats.\n3-operand format # General form: destination ← operand1 OP operand2\nMPY T1, C, D ; T1 = C × D ADD T2, B, T1 ; T2 = B + T1 (= B + C × D) DIV T3, E, F ; T3 = E / F SUB T4, T2, T3 ; T4 = T2 − T3 (= B + C×D − E/F) MPY X, A, T4 ; X = A × T4 (final result) Total: 5 instructions. Each instruction is self-contained: it specifies input, operation, and destination. The instructions are longer (three addresses encoded in each one), but the program is compact.\n2-operand format # General form: Op1 ← Op1 OP Op2 (the first operand is also the destination).\nMOV X, C MPY X, D ; X = C × D ADD X, B ; X = B + C × D MOV T, E DIV T, F ; T = E / F SUB X, T ; X = B + C×D − E/F MPY X, A ; X = A × (B + C×D − E/F) Total: 7 instructions. We need to insert MOVs to preserve original values before overwriting them. This is the format of the classic x86 family, based on 2-operand instructions (extensions such as AVX added a 3-operand form).\n1-operand format (accumulator) # General form: ACC ← ACC OP Operand. All operations go through a special register called the accumulator.\nLDA C ; ACC = C MPY D ; ACC = C × D ADD B ; ACC = B + C × D STR T ; T = ACC (saves temporary) LDA E ; ACC = E DIV F ; ACC = E / F STR T2 ; T2 = ACC LDA T ; ACC = T (= B + C × D) SUB T2 ; ACC = B + C×D − E/F MPY A ; ACC = A × (B + C×D − E/F) STR X ; X = ACC (final result) Total: 11 instructions. Each instruction is compact (a single address), but more of them are needed, and more memory accesses to save intermediate results. This format was common in the first microprocessors.\nComparison # Format No. of instructions Space per instruction Typical use 3 operands 5 Large (3 addresses) Modern RISC architectures 2 operands 7 Medium (2 addresses) Classic x86 1 operand (ACC) 11 Small (1 address) First microprocessors More than the number of lines, the hardware designer looks at the balance between bits and cycles. It is worth putting numbers on this. Suppose an 8-bit opcode and 20-bit memory addresses, plausible parameters for a teaching machine. Then:\na 3-operand instruction takes \\(8 + 3 \\times 20 = 68\\) bits; a 2-operand one, \\(8 + 2 \\times 20 = 48\\) bits; a 1-operand one, \\(8 + 20 = 28\\) bits. The 3-operand instruction is \u0026ldquo;heavier\u0026rdquo;, but it solves the computation with fewer trips to memory. The 1-operand instruction is \u0026ldquo;light\u0026rdquo;, but it requires many more accesses to constantly load and save the accumulator. It is the eternal dilemma between spending memory on the code or spending time on execution, and the two sides of the scale are measured in different units, which makes the decision hard.\nA real format: the RISC-V R-type # The numbers above come from a paper machine. It is worth seeing how a real ISA solves the problem, and RISC-V is the best choice for that: besides being open and documented, it has a very regular encoding, with rs1, rs2, and rd in the same position in every format. Its three-operand format, called R-type, fits in 32 bits:\nField Bits Function funct7 7 Completes the identification of the operation rs2 5 Second source register rs1 5 First source register funct3 3 Completes the identification of the operation rd 5 Destination register opcode 7 Identifies the instruction class Three 5-bit fields because the architecture has \\(2^5 = 32\\) registers. Since the three operands are registers, not memory addresses, together they take only 15 bits: the whole instruction fits in 32, 17 are left to identify the operation, and the total is far from the 68 bits of the three-memory-address format in the previous computation.\nThe figure below puts the four formats on the same bit ruler, the three from the paper machine and the R-type. The RISC-V three-operand format comes out shorter than the two-operand one of the hypothetical machine, and only 4 bits longer than the one-operand one.\nWith an 8-bit opcode and 20-bit addresses, the 3-, 2-, and 1-operand instructions take 68, 48, and 28 bits. The RISC-V R-type also has three operands and takes 32, because each operand is a 5-bit register number. This is the difference that the RISC × CISC discussion usually summarizes poorly: it is not \u0026ldquo;fewer instructions\u0026rdquo;, it is operands in registers and a fixed-size format, which together make decoding trivial and the pipeline predictable.\nAn add rd, rs1, rs2 in RISC-V has opcode = 0110011, funct3 = 000, and funct7 = 0000000; changing funct7 to 0100000 turns the same instruction into sub. The operation changes, the format does not, and that is why the decoder of a RISC processor fits in much less silicon than that of an x86, where an instruction can have from 1 to 15 bytes.\nStrategy for counting the number of instructions 3 operands: each operation in the expression generally produces one instruction with an explicit destination. In the expression \\[ X = A \\times (B + C \\times D - E/F) \\] there are 5 arithmetic operations: one multiplication \\(C \\times D\\), one addition, one division, one subtraction, and the final multiplication by \\(A\\). Therefore, the 3-operand program uses 5 instructions, with no need for an extra \u0026ldquo;assignment\u0026rdquo; instruction.\n2 operands: add MOVs when you need to preserve a value before overwriting it.\n1 operand: add LDA and STR whenever you need to load data into the accumulator and save intermediate results.\nInstructions such as SQRT, NEG, or immediate shifts can appear in real or hypothetical ISAs, but their presence alone does not define an architecture as CISC or RISC.\nThe distinction between RISC and CISC involves a larger set of design decisions:\nregularity of the instruction formats; number of addressing modes; average complexity of the instructions; ease of pipelining and decoding. Therefore, the following example is just an example of an extended hypothetical ISA, and not a proof of a \u0026ldquo;CISC philosophy\u0026rdquo;.\nExample: compute \\(X_1 = (-B + \\sqrt{B^2 - 4C}) / 2\\) with 1 operand Using 1-operand (accumulator) instructions with direct and immediate addressing, and with the special instruction SQRT for the square root:\nThe order of operations here requires care. With a single accumulator, \\(B^2 - 4C\\) cannot be computed in the obvious sequence: subtracting 4 and then multiplying by \\(C\\) would produce \\((B^2 - 4) \\times C\\), which is a different computation. The way is to compute \\(4C\\) first, store it in a temporary, and only then subtract:\nLDA C ; ACC = C MPY #4 ; ACC = 4C (immediate addressing) STR TEMP ; TEMP = 4C LDA B ; ACC = B MPY B ; ACC = B² SUB TEMP ; ACC = B² − 4C SQRT ; ACC = √(B² − 4C) STR TEMP ; TEMP = √(B² − 4C) (reuses the temporary) LDA B ; ACC = B NEG ; ACC = −B ADD TEMP ; ACC = −B + √(B² − 4C) SHR #1 ; ACC = ACC / 2 (shift right = divide by 2) STR X1 ; X1 = ACC Thirteen instructions and one temporary address for an expression that, on paper, has five operations. This is the cost of the single accumulator: every intermediate result that cannot be consumed immediately has to be parked in memory.\nThis formula has a problem that assembly does not solve The order chosen here is arithmetically correct, but the quadratic formula written this way is numerically unstable in floating point: when \\(B^2\\) is much larger than \\(4C\\), the root is almost equal to \\(|B|\\), and the subtraction of two close numbers destroys the significant digits. This is catastrophic cancellation, discussed in detail in the article on IEEE 754 from the sister series, along with the alternative form that avoids the problem. It is worth mentioning because it is a good example of something this article cannot see: the instruction is right, the encoding is right, and the result can still be wrong, because the representation of numbers is a layer below the representation of instructions.\nAddressing modes # The addressing modes define how the processor obtains the operand of an instruction.\nThe central point is this: the effective address (EA) is the final address of the operand in memory, when the instruction actually accesses memory. In some modes there is no memory EA, because the data is already in the instruction itself or in a register.\n\u0026#x2197; flowchart TD A([Instruction]) --\u003e B{Addressing\nMode} B --\u003e C[\"Immediate\noperand = constant\"] B --\u003e D[\"Direct\nEA = address\"] B --\u003e E[\"Indirect\nEA = M[address]\"] B --\u003e F[\"Reg. Direct\noperand = Rn\"] B --\u003e G[\"Reg. Indirect\nEA = Rn\"] B --\u003e H[\"Indexed\nEA = address + Ri\"] B --\u003e I[\"Base + Displacement\nEA = RB + disp\"] After computing the EA, when there is a memory access, the operand will be:\n$$ \\text{Operand} = M[EA] $$For the processor, there are two ways to identify which of these modes to use: either the Opcode already carries that information (the code 1010 means direct LDA and 1011 means indirect LDA) or the instruction reserves a specific bit field just to signal the addressing mode to the decoder.\nImmediate mode # The operand is the constant encoded directly in the instruction: it is not an address, it is the value itself.\n$$\\text{Operand} = \\text{constant in the instruction}$$ Advantage: very fast, with no additional MM access Disadvantage: limited by the field size; only serves for small, fixed constants Example: LDA #5 → ACC = 5 Direct (absolute) mode # The instruction field contains the memory address where the operand is.\n$$ EA = \\text{address} \\qquad\\Rightarrow\\qquad \\text{Operand} = M[EA] $$ Advantage: simple and direct; easy to understand and debug Disadvantage: the address is fixed in the object code, which makes relocating the program harder Example: LDA 100 → ACC = M[100] Indirect mode # The field contains an address that holds the address of the operand, an extra level of indirection. The field points to a pointer.\n$$ EA = M[\\text{address}] \\qquad\\Rightarrow\\qquad \\text{Operand} = M[EA] $$ Advantage: flexible for pointers and dynamic structures Disadvantage: requires 2 MM accesses, one to fetch the address and another to fetch the data Example: LDA @100 → ACC = M[M[100]] Indirect mode: 2 memory accesses Indirect mode is often confused with register indirect mode. The difference is in the number of MM accesses:\nIndirect (memory): 2 accesses. The pointer\u0026rsquo;s address is in memory, so you must fetch the pointer (1st access) and then the data (2nd access) Register indirect: 1 access. The pointer is already in a register (no access to fetch it), so there is only the access to the data Register direct mode # The operand is in a processor register. Here there is no effective address: there is nothing to compute or to fetch from memory, because the data is already inside the CPU. The operand field in the instruction does not hold an address, it holds the register number:\n$$\\text{Operand} = R_n \\qquad (\\text{without } EA)$$ Advantage: the fastest possible, since reading a register costs about 1 cycle, against 100 to 300 cycles for a main memory access that misses the cache. The difference is two orders of magnitude.\nDisadvantage: limited number of registers; programs need to manage which data they keep in registers\nExample: ADD R1, R2 → R1 = R1 + R2\nRegister indirect mode # The register contains the memory address where the operand is. The register works as a pointer.\n$$ EA = R_n \\qquad\\Rightarrow\\qquad \\text{Operand} = M[EA] $$ Advantage: very flexible for traversing arrays using a register as an incrementable pointer Example: LDA (R1) → ACC = M[R1] Indexed mode # The base address is encoded in the instruction; the index register \\(R_i\\) is added to it to compute the effective address.\n$$ EA = \\text{address} + R_i \\qquad\\Rightarrow\\qquad \\text{Operand} = M[EA] $$ Advantage: ideal for traversing arrays sequentially, because you just increment \\(R_i\\) on each iteration Example: LDA 100(R1) → ACC = M[100 + R1] Imagine the task of summing 100 numbers in an array. In direct mode, you would have to write the same instruction 100 times, changing only the address. With indexed mode, you write the instruction once, pointing to the base of the array, and use an index register to \u0026ldquo;walk\u0026rdquo; through the elements. It is the difference between 300 lines of code and an efficient 5-line loop.\nBase + displacement mode # The register contains the base; the instruction carries a fixed displacement.\n$$ EA = R_B + \\text{displacement} \\qquad\\Rightarrow\\qquad \\text{Operand} = M[EA] $$ Conceptual example: accessing a structure field or a local variable on the stack Typical use: relocatable code, stack, function activation, access to struct fields This mode is very common in modern architectures because it cleanly separates the dynamic base address from the fixed displacement known at compile time.\nModern examples of base + displacement In current architectures, this mode shows up all the time.\nx86-64: mov eax, [rbp-8] Accesses a local variable from the stack base pointer.\nAArch64: ldr x0, [x1, #16] Accesses the memory location whose address is x1 + 16.\nThis makes base + displacement mode easier to recognize in real code than old examples based on historical segmentation.\nIndexed × base + displacement In both cases, the final address is the sum of two values. The difference is:\nIndexed: base in the instruction + index in the register Ideal for arrays and iteration.\nBase + displacement: base in the register + displacement in the instruction Ideal for the stack, structs, and relocatable code.\nComparison table # Mode EA Operand MM accesses to get the data Main advantage Immediate (none) value in the instruction 0 Constants; maximum speed Direct \\(\\text{address}\\) \\(M[EA]\\) 1 Simple; global variables Indirect \\(M[\\text{address}]\\) \\(M[EA]\\) 2 Pointers in memory Reg. Direct (none) \\(R_n\\) 0 Fast operation on registers Reg. Indirect \\(R_n\\) \\(M[EA]\\) 1 Pointer in a register Indexed \\(\\text{address} + R_i\\) \\(M[EA]\\) 1 Traversing arrays Base + Displacement \\(R_B + \\text{disp}\\) \\(M[EA]\\) 1 Stack, structs, relocation Example: addressing modes with a memory table Consider the memory:\nAddress Contents 1AC1 1AC6 1AC2 1AC4 1AC3 1AC1 1AC4 1AC3 1AC5 1AC1 1AC6 1AC5 a) Direct, operand = 1AC1\n$$ EA = 1AC1 \\qquad\\Rightarrow\\qquad ACC \\leftarrow M[1AC1] = \\mathbf{1AC6} $$b) Immediate, operand = 1AC5\nThe operand field already contains the value:\n$$ ACC \\leftarrow \\mathbf{1AC5} $$c) Indirect, operand = 1AC3\nFirst the effective address is computed:\n$$ EA = M[1AC3] = 1AC1 $$Then the operand is fetched:\n$$ ACC \\leftarrow M[EA] = M[1AC1] = \\mathbf{1AC6} $$d) If the contents of 1AC1 change to 0\nDirect: \\(ACC = M[1AC1] = \\mathbf{0}\\) Immediate: \\(ACC = \\mathbf{1AC5}\\) (unchanged) Indirect: \\(EA = M[1AC3] = 1AC1 \\Rightarrow ACC = M[1AC1] = \\mathbf{0}\\) Immediate mode is the only one that does not depend on the current contents of memory.\nPython bonus: computing effective addresses # Understanding hardware by reading diagrams and mathematical formulas can seem very abstract at first. But, in the end, addressing modes are just pointer calculation logic. For a programmer, the easiest way to demystify what the silicon does is to translate that mechanics into code. See how the hardware would compute effective addresses using simple Python functions:\n# Simulates the main addressing modes. # Mind the convention: each function returns the OPERAND (the data the # instruction wanted) and the number of MM accesses that were needed, not # the effective address. The EA shows up along the way, commented where # relevant. def ea_immediate(value): \u0026#34;\u0026#34;\u0026#34;Immediate mode: the operand is the value itself.\u0026#34;\u0026#34;\u0026#34; return value, 0 # (data, MM accesses) def ea_direct(memory: dict, address: int): \u0026#34;\u0026#34;\u0026#34;Direct mode: EA = address; operand = M[EA].\u0026#34;\u0026#34;\u0026#34; return memory[address], 1 def ea_indirect(memory: dict, address: int): \u0026#34;\u0026#34;\u0026#34;Indirect mode: EA = M[address]; operand = M[EA].\u0026#34;\u0026#34;\u0026#34; ptr = memory[address] # 1st access: fetches the pointer (= EA) return memory[ptr], 2 # 2nd access: fetches the data def ea_reg_indirect(memory: dict, register: int): \u0026#34;\u0026#34;\u0026#34;Register indirect mode: EA = Rn; operand = M[EA]. The parameter receives the CONTENTS of the register, which is already the address. \u0026#34;\u0026#34;\u0026#34; return memory[register], 1 def ea_indexed(memory: dict, base: int, index: int): \u0026#34;\u0026#34;\u0026#34;Indexed mode: EA = base + Ri; operand = M[EA].\u0026#34;\u0026#34;\u0026#34; return memory[base + index], 1 def ea_base_displacement(memory: dict, rb: int, displacement: int): \u0026#34;\u0026#34;\u0026#34;Base + displacement mode: EA = RB + displacement; operand = M[EA].\u0026#34;\u0026#34;\u0026#34; return memory[rb + displacement], 1 # Memory table from the previous example mem = { 0x1AC1: 0x1AC6, 0x1AC2: 0x1AC4, 0x1AC3: 0x1AC1, 0x1AC4: 0x1AC3, 0x1AC5: 0x1AC1, 0x1AC6: 0x1AC5, } data, accesses = ea_direct(mem, 0x1AC1) print(f\u0026#34;Direct (1AC1): {data:04X} ({accesses} MM access(es))\u0026#34;) # Direct (1AC1): 1AC6 (1 MM access(es)) data, accesses = ea_immediate(0x1AC5) print(f\u0026#34;Immediate(1AC5): {data:04X} ({accesses} MM access(es))\u0026#34;) # Immediate(1AC5): 1AC5 (0 MM access(es)) data, accesses = ea_indirect(mem, 0x1AC3) print(f\u0026#34;Indirect (1AC3): {data:04X} ({accesses} MM access(es))\u0026#34;) # Indirect (1AC3): 1AC6 (2 MM access(es)) # Array with indexed mode array = {100: 10, 101: 20, 102: 30, 103: 40} for i in range(4): val, _ = ea_indexed(array, 100, i) print(f\u0026#34;array[{i}] = {val}\u0026#34;) # array[0] = 10, array[1] = 20, array[2] = 30, array[3] = 40 Conclusion and next articles # The ISA defines the vocabulary the processor understands; instruction formats and addressing modes are the grammar of that vocabulary. With three operands per instruction and seven addressing modes, a processor can express anything from simple additions to dynamic access to complex data structures. The choice between more operands (CISC) and fewer operands (RISC) reflects a trade-off between expressiveness and regularity that modern hardware resolves in sophisticated ways, the topic of the article Advanced Architectures, at the end of the series.\nIf you want to see real instructions being generated right now, the Compiler Explorer is the right tool: you write C, C++, Rust, or a dozen other languages on one side and see, on the other, the assembly the compiler produces, and you can choose the architecture, including RISC-V, and compare the formats this article described. It is the fastest way to get out of the table and down to the bit.\nIn the next article, we will follow the life cycle of a program: from source code in a high-level language, through compilation, linking, and loading, until it becomes the machine instructions the processor executes. We will see how the ISA is the bridge between the code we write and the silicon that executes it, and how ISA design decisions impact the efficiency and flexibility of programs.\nSee you then!\n","date":"2026-04-30","externalUrl":null,"permalink":"/2026/04/instruction-representation/","section":"Blog","summary":"The same expression costs 5 instructions with three operands, 7 with two, and 11 with an accumulator. We will see the ISA, the contract between hardware and software, instruction formats, the RISC-V R-type, and the seven addressing modes that tell where each operand is.","title":"Instruction Representation: ISA, Formats, and Addressing Modes","type":"posts"},{"content":"","date":"2026-04-30","externalUrl":null,"permalink":"/tags/isa/","section":"tags","summary":"","title":"ISA","type":"tags"},{"content":"","date":"2026-04-28","externalUrl":null,"permalink":"/tags/ascii/","section":"tags","summary":"","title":"ASCII","type":"tags"},{"content":"","date":"2026-04-28","externalUrl":null,"permalink":"/tags/binario/","section":"tags","summary":"","title":"binário","type":"tags"},{"content":"","date":"2026-04-28","externalUrl":null,"permalink":"/tags/binary/","section":"tags","summary":"","title":"binary","type":"tags"},{"content":" In the previous article, Pipeline and Buses, we saw how the CPU executes multiple instructions in parallel and how buses carry information between components. But what is that information? To the hardware, everything is bits: high and low voltages, zeros and ones. The difference between a character, a negative integer, and a floating-point number lies exclusively in the interpretation convention, in the way we agreed that a bit pattern should be read.\nThis choice also shapes the physical design of the chip. The number of bits we choose to represent a piece of data directly dictates the width of the data bus, the capacity of the registers, and the complexity of the ALU. The first of these conventions we need to understand does not deal with heavy math, but with the way the computer learned to read and write with us: text.\nThis article walks through these fundamental conventions: the standards for representing text (ASCII, Unicode, UTF-8), the three methods for signed integers (and why two\u0026rsquo;s complement took over), bit shifts, and the IEEE 754 standard for real numbers. Understanding these representations is essential to understand why 0.1 + 0.2 ≠ 0.3 in programming and how negative numbers work in hardware.\nIf you have already read \u0026ldquo;From Zero to Float\u0026rdquo;, you can skim this Much of this article revisits what the series From Zero to Float covers in much more depth: number bases, two\u0026rsquo;s complement, overflow, shifts, and IEEE 754. The repetition is deliberate: this series needs to be self-contained, and readers who arrived here directly should not have to leave to understand the rest. But if you come from there, the useful reading is selective: the sections on characters and on the representation of instructions in memory are new, and the rest works as a quick review.\nThis article, then, serves as an integrated summary of these concepts, focused on the computer architecture perspective.\nCharacter representation # How do we represent letters, digits, and symbols as bits? For that, the computer uses a character encoding: a convention that associates symbols with numbers.\nIt is important to separate three ideas that sometimes show up mixed together:\ncharacter code: the table of symbols; code point: the abstract number associated with a character; encoding: the concrete way of storing that code point in bytes. System Nature Note ASCII 7-bit character code 128 symbols; historical basis of text in modern systems EBCDIC 8-bit character code Historically used in IBM environments Latin-1 (ISO 8859-1) 8-bit character code Extension for Western languages Unicode Universal repertoire of code points Defines the characters; does not define by itself how to store them UTF-8 / UTF-16 / UTF-32 Unicode encodings Ways of storing code points in bytes ASCII, Latin-1, and the 8th bit # ASCII (American Standard Code for Information Interchange) uses 7 bits and represents 128 symbols. It includes:\ncontrol characters (0 to 31 and 127, the DEL); printable characters (32 to 126), such as letters, digits, and punctuation. Historically, in many systems the 8th bit of the byte could be used as a parity bit to detect transmission errors. In other contexts, that bit came to be used for 8-bit extensions, such as Latin-1, making it possible to represent accented characters and additional symbols.\nIt is worth seeing the table at work with concrete numbers, because this is where the encoding stops being an abstraction:\nCharacter Decimal Hexadecimal Binary '0' 48 0x30 0011 0000 '9' 57 0x39 0011 1001 'A' 65 0x41 0100 0001 'Z' 90 0x5A 0101 1010 'a' 97 0x61 0110 0001 'z' 122 0x7A 0111 1010 Two things jump out of this table. The first is that digits start at 0x30, which turns the conversion from character to number into a subtraction: '7' - '0' = 7. The second is prettier, and it is exactly the theme of this article:\n$$\\text{'a'} - \\text{'A'} = 97 - 65 = 32 = 2^5$$Uppercase and lowercase versions of the same character differ by a single bit: bit 5. 'A' is 0100 0001 and 'a' is 0110 0001; the bit of weight 32 changes and nothing else. That is why changing the case of an ASCII letter needs no table at all, only a bitwise operation: c ^ 0x20 toggles between uppercase and lowercase, c | 0x20 forces lowercase, and c \u0026amp; 0xDF forces uppercase. The ASCII table was not ordered by chance; this regularity was designed.\nUnicode and the UTF encodings # Unicode should not be understood as a fixed 16-bit encoding, but as a universal repertoire of code points. Its code space goes from U+0000 to U+10FFFF.\nTo store these code points in memory or in files, we use encodings such as:\nEncoding Length Main feature UTF-8 1 to 4 bytes ASCII-compatible in the first 128 codes UTF-16 2 or 4 bytes Uses one 16-bit unit in the BMP and surrogate pairs outside it UTF-32 4 bytes One code point per 32-bit unit So the right way to think about it is:\nUnicode = repertoire of characters; UTF-8 / UTF-16 / UTF-32 = ways of encoding that repertoire. UTF-8 dominates modern practice UTF-8 became the dominant encoding because it preserves full compatibility with ASCII in the first 128 codes and uses variable length only when necessary. For text files, protocols, and web pages, it is today the default choice.\nHow UTF-8 organizes the bits # Saying that UTF-8 uses \u0026ldquo;1 to 4 bytes\u0026rdquo; leaves out the interesting part: how the program knows how many bytes to read. The scheme is a convention of leading bits, and it is the same bit-field idea that the whole article has been dealing with:\nCode point range Bytes Bit pattern U+0000 to U+007F 1 0xxxxxxx U+0080 to U+07FF 2 110xxxxx 10xxxxxx U+0800 to U+FFFF 3 1110xxxx 10xxxxxx 10xxxxxx U+10000 to U+10FFFF 4 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx The x are the bits of the code point, filled from right to left. The rest is signaling: a byte that starts with 0 is pure ASCII; one that starts with 110 announces \u0026ldquo;I am the first of two\u0026rdquo;; and every continuation byte starts with 10.\nFrom this comes the property that made UTF-8 win: it is self-synchronizing. Since no continuation byte can be mistaken for a leading byte, a program that lands in the middle of a text can find the start of the next character by walking backward until it finds a byte that does not start with 10. Many earlier variable-length encodings, such as Shift-JIS, did not have this.\nAn example with ç, code point U+00E7:\nU+00E7 in binary is 1110 0111, with 8 bits, so it falls in the 2-byte range; the 11 bits of the template receive 000 1110 0111; first byte: 110 + 00011 = 1100 0011 = 0xC3; second byte: 10 + 100111 = 1010 0111 = 0xA7. That is, ç in UTF-8 is the bytes 0xC3 0xA7, and that is why a UTF-8 file read as Latin-1 shows ç in place of ç: the two bytes show up as two separate characters.\nFor the full story of this confusion, the canonical reference is The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!), by Joel Spolsky.\nUnsigned integers # For unsigned integers, the representation is direct: each bit corresponds to a power of 2, from the LSB (least significant bit, \\(b_0\\)) to the MSB (most significant bit):\n$$N_{10} = \\sum_{i=0}^{n-1} b_i \\cdot 2^i$$Example: \\(10110101_2\\):\n$$= 1\\cdot2^7 + 0\\cdot2^6 + 1\\cdot2^5 + 1\\cdot2^4 + 0\\cdot2^3 + 1\\cdot2^2 + 0\\cdot2^1 + 1\\cdot2^0 = 128 + 32 + 16 + 4 + 1 = 181_{10}$$ Binary ↔ hexadecimal conversion # Hexadecimal works as a compact notation for binary: group the bits in fours, from right to left, and convert each group to its equivalent hexadecimal digit (0 to 9, A to F).\nBinary Groups of 4 Hexadecimal Decimal 01011010 0101 + 1010 5A 90 11110011 1111 + 0011 F3 243 Check: \\(\\text{5A}_{16} = 5 \\times 16 + 10 = 80 + 10 = 90_{10}\\) ✓\nSigned integers # Representing negative numbers requires an additional convention. Throughout history, three methods have been used:\nSign and magnitude # The MSB indicates the sign (0 = positive, 1 = negative); the remaining \\(n-1\\) bits represent the magnitude. Range: \\(-(2^{n-1}-1)\\) to \\(+(2^{n-1}-1)\\). The fatal problem: two zeros (\\(+0 = 0000\\ 0000\\) and \\(-0 = 1000\\ 0000\\)) complicate comparison and arithmetic circuits.\nOne\u0026rsquo;s complement (C1) # The negative of a number is obtained by inverting all the bits. The result for \\(-5\\) in 8 bits: \\(+5 = 0000\\ 0101 \\Rightarrow -5 = 1111\\ 1010\\). The arithmetic becomes simpler than in sign and magnitude, but the two-zeros problem remains (\\(0000\\ 0000\\) and \\(1111\\ 1111\\)). C1 is rarely used in modern systems.\nTwo\u0026rsquo;s complement (C2): the modern standard # The negative is obtained by inverting all the bits and adding 1 to the result. C2 has only one zero, and subtraction reduces to addition: the ALU hardware needs only one adder to perform both operations.\n$$\\text{Range:} \\quad -2^{n-1} \\leq x \\leq +(2^{n-1}-1)$$ \\(n\\) bits Minimum Maximum 8 \\(-128\\) \\(+127\\) 16 \\(-32{,}768\\) \\(+32{,}767\\) 32 \\(-2{,}147{,}483{,}648\\) \\(+2{,}147{,}483{,}647\\) With the three methods defined, it is worth seeing them read exactly the same bits. The figure below does this with three bytes. The first, positive, gives the same value in all three schemes. The second shows the central point of this article: the same byte becomes three different numbers, because the value is not in the bits, it is in the convention used to read them. And the third, 1000 0000, is the edge case: a repeated zero in sign and magnitude, the smallest negative of one\u0026rsquo;s complement, and the \\(-128\\) that only two\u0026rsquo;s complement can represent.\nThree bytes, each read by the three signed integer schemes. The positives coincide; the negatives diverge; and 1000 0000 is worth −0, −127, or −128 depending on the convention. Why is two\u0026rsquo;s complement the standard? Four concrete reasons make C2 the universal standard:\nSubtraction = addition: \\(A - B = A + C2(B)\\), the ALU needs only an adder. Single zero: no ambiguity between \\(+0\\) and \\(-0\\). Trivial sign extension: to take a number from 8 to 16 bits, just repeat the sign bit on the left. \\(11111011\\) (\\(-5\\)) becomes \\(1111111111111011\\), and is still worth \\(-5\\). One\u0026rsquo;s complement also has this property; in sign and magnitude, extending requires moving the sign bit. Comparison with the same circuits as the unsigned integer: adding, subtracting, and comparing magnitudes use exactly the same hardware as the unsigned version, and the only thing that changes is the interpretation of the flags. There is no second set of circuits for signed numbers. The asymmetric range (\\(-128\\) to \\(+127\\) in 8 bits) is sometimes listed as yet another advantage, but it is not: it is a consequence of having only one zero. With \\(2^8\\) patterns and one of them spent on zero, 255 remain to be distributed among positives and negatives, and the split cannot be symmetric. And far from being an advantage, it is the origin of a classic bug: \\(-128\\) has no representable opposite, so negating \\(-128\\) in 8 bits returns \\(-128\\).\nOperations in two\u0026rsquo;s complement # How to compute the C2 of a number (negation) # The algorithm has three steps:\n1 Write in binary Write the number in binary. 2 Invert the bits Invert all the bits, obtaining the one\u0026rsquo;s complement. 3 Add 1 Add 1 to the result. Step Bits \\(+5\\) in binary 0000 0101 Invert all the bits (C1) 1111 1010 Add 1 → result 1111 1011 = \\(-5\\) ✓ Check (\\(+5 + (-5)\\) must be zero):\n$$0000\\ 0101 + 1111\\ 1011 = \\underbrace{1}_{\\text{carry}}\\ 0000\\ 0000 \\xrightarrow{\\text{discard carry}} 0000\\ 0000 = 0 \\ \\checkmark$$ Subtraction as addition: \\(A - B = A + C2(B)\\) # To understand how this works in practice, let us simulate the operation 5 - 3. Following the formula above, we set \\(A = 5\\) and \\(B = 3\\). Instead of using a dedicated circuit to subtract, the CPU computes the two\u0026rsquo;s complement of 3 (obtaining the bit representation of -3) and adds that value directly to the number 5. The following table details this process step by step using 8 bits:\nStep Operation Result Represent \\(+3\\) binary 0000 0011 Invert bits (C1) 1111 1100 Add 1 → \\(C2(3) = -3\\) 1111 1101 Add \\(5 + (-3)\\) 0000 0101 + 1111 1101 \\(\\underbrace{1}_{\\text{carry}}\\ 0000\\ 0010\\) Discard carry 0000 0010 = \\(2\\) ✓ Example: addition, subtraction, and overflow in 8 bits Consider \\(A = 00110010_2 = +50_{10}\\) and \\(B = 11010000_2\\). To determine the value of B: since the MSB is 1, it is negative. \\(C2(B) = 00110000_2 = 48\\), therefore \\(B = -48_{10}\\).\na) \\(A + B\\):\n\\(00110010 + 11010000 = 1\\ 00000010_2\\) → discard carry → \\(00000010_2 = +2_{10}\\) ✓\n(\\(50 + (-48) = +2\\); opposite signs never generate overflow)\nb) \\(B - A = B + C2(A)\\):\n\\(C2(A)\\): invert \\(00110010 = 11001101\\) + 1 = \\(11001110_2\\)\n\\(11010000 + 11001110 = 1\\ 10011110_2\\) → discard carry → \\(10011110_2\\)\nMSB=1 → negative: \\(C2(10011110) = 01100010 = 98\\) → result = \\(-98_{10}\\)\n(\\(-48 - 50 = -98\\) ✓, within the range \\([-128, +127]\\))\nc) Can two positives add up to a negative result?\nYes: that is exactly overflow. Example: \\(C = 01000001_2 = +65\\) and \\(D = 01100000_2 = +96\\):\n\\(C + D = 10100001_2\\) → MSB=1 → interpreted as negative (\\(-95\\)), but \\(65 + 96 = 161 \u003e 127\\) → overflow ❌\nDetecting overflow # Overflow in two\u0026rsquo;s complement Overflow occurs when the correct result does not fit in the available number of bits:\nTwo positives added → negative result = overflow ❌ Two negatives added → positive result = overflow ❌ Positive + Negative → never generates overflow ✓ Hardware rule: overflow occurs when the carry into the sign bit is different from the carry out of the sign bit.\nIf this part about overflow was hard for you, we have a whole article dedicated to it in the series From Zero to Float, where we explain the concept of overflow in detail, with examples and exercises.\nBit shifts # Two\u0026rsquo;s complement also lets the hardware use mathematical \u0026ldquo;shortcuts\u0026rdquo;. The most efficient possible operation to multiply or divide by powers of 2 does not engage complex adders, but rather bit shifts: an operation solved by \u0026ldquo;pushing\u0026rdquo; bits sideways in a single ALU cycle.\nOperation Mathematical effect Bit inserted Left shift (1 bit) Multiplies by 2 0 on the right Logical right shift (1 bit) Divides by 2 (unsigned integer) 0 on the left Arithmetic right shift (1 bit) Divides by 2 (signed integer) Copy of the MSB (sign bit) The MSB copy works as follows: if the number is positive (MSB = 0), the right shift fills with 0, keeping the result positive. If the number is negative (MSB = 1), the shift fills with 1, preserving the negative sign. This makes the arithmetic right shift behave like an integer division by 2, rounding down, even for negative numbers. For example: \\(-3\\) in 8 bits is 1111 1101. Shifting right arithmetically gives 1111 1110, because the MSB is 1, indicating a negative number, and 1 is inserted on the left to keep the sign. The result is \\(-2\\), which is the expected result of \\(-3\\) divided by 2, rounding down.\nExamples in 8 bits:\nOperation Before After Value Left shift 0000 0011 (3) 0000 0110 6 = 3 × 2 ✓ Logical right shift 0000 0110 (6) 0000 0011 3 = 6 ÷ 2 ✓ Arithmetic right shift 1111 0000 (−16) 1111 1000 −8 = −16 ÷ 2 ✓ Arithmetic right shift: careful with the interpretation # The arithmetic right shift is often presented as a quick way to \u0026ldquo;divide by 2\u0026rdquo; signed integers. This is useful as intuition, but deserves a remark:\nfor unsigned integers, the logical right shift is equivalent to integer division by 2; for signed integers in two\u0026rsquo;s complement, the arithmetic right shift preserves the sign bit, which makes the result behave like a rounding down in many negative cases. So it is worth stating carefully: the arithmetic right shift is a hardware operation that, in two\u0026rsquo;s complement, usually corresponds to integer division by a power of 2, preserving the sign. \u0026ldquo;Usually\u0026rdquo; because the rounding is not the same as the language\u0026rsquo;s division: the shift rounds down (toward \\(-\\infty\\)), while the integer division operator of many languages rounds toward zero. For \\(-13\\) in 8 bits, the 2-bit shift gives \\(-4\\); in C, -13 / 4 gives \\(-3\\).\nTip: arithmetic vs. logical Use arithmetic shift right to divide signed integers (it copies the sign bit, preserving it). Use logical shift for unsigned integers (it fills with 0). Shifting left multiplies by 2 in both cases: just make sure there is no overflow.\nExample: repeated division by 2 with arithmetic shift Consider \\(A = 01011110_2 = +94_{10}\\) and \\(B = 10011101_2 = -99_{10}\\).\nDividing A (positive) by 2 with arithmetic shift right:\n\\(01011110 \\to 00101111 = 47 \\to 00010111 = 23 \\to 00001011 = 11 \\to 00000101 = 5 \\to 00000010 = 2 \\to 00000001 = 1 \\to 00000000 = 0\\)\nEach step divides by 2 (truncating), until reaching zero.\nDividing B (negative) by 2: the arithmetic shift copies the sign bit (MSB=1), preserving the negative sign:\n\\(10011101 \\to 11001110 = -50 \\to 11100111 = -25 \\to 11110011 = -13 \\to 11111001 = -7 \\to 11111100 = -4 \\to 11111110 = -2 \\to 11111111 = -1 \\to 11111111 = -1\\)\nFor negative numbers, the process converges to −1 (not to zero), because the sign bit is never eliminated.\nFloating point: the IEEE 754 standard # Although two\u0026rsquo;s complement integers are perfect for exact counting and memory manipulation, the physical world (with its microscopic distances and astronomical quantities) demands a flexibility that fixed bits cannot offer. For continuous quantities, we need approximate real numbers. Floating point solves this by using a normalized scientific notation in base 2:\n$$(-1)^S \\times 1{.}\\text{significand} \\times 2^E$$where \\(S\\) is the sign bit (0 = positive, 1 = negative), \\(E\\) is the integer exponent, and the significand is the fractional bits after the implicit 1..\nBut where do the rules for storing this in bits come from? Historically, each computer manufacturer created its own way of representing floating-point numbers, which caused chaos: the same scientific calculation gave different results depending on which brand of computer ran it. To unify the industry, the IEEE (Institute of Electrical and Electronics Engineers, the world\u0026rsquo;s main standardization organization for electrical engineering and computing) published the IEEE 754 standard in 1985.\nThe original IEEE 754 standard defined exactly the basic single and double precision formats, establishing only minimum requirements for \u0026ldquo;extended\u0026rdquo; formats. The following table shows the 32-bit and 64-bit formats defined in the standard, in addition to the famous 80-bit extended format (an Intel/x86-specific implementation that meets the standard\u0026rsquo;s extended format requirements):\nPrecision Sign Exponent Significand Total Bias Single (32 bits) 1 bit 8 bits 23 bits 32 bits 127 Double (64 bits) 1 bit 11 bits 52 bits 64 bits 1,023 Extended (80 bits) 1 bit 15 bits 64 bits 80 bits 16,383 The exponent is stored with a bias: the real value is the stored one minus the bias, which allows representing negative exponents without a sign bit of their own. The 1 bit to the left of the point is implicit in single and double: it does not need to be stored, which adds an extra bit of effective precision (24 bits in single, 53 bits in double). In the extended format (80 bits), on the other hand, this leading integer bit is explicit and must be stored, taking up real space and resulting in exactly 64 bits of precision in the significand. Nowadays, the 80-bit extended format is widely considered legacy: it was the internal standard of the old math coprocessors (x87 FPU) of the x86 architecture. Modern processors generally perform floating-point calculations using vector registers (SSE/AVX) optimized for the single and double formats. However, extended is still supported on current processors for backward compatibility (often being mapped to the long double type in C/C++ compilers on the x86 architecture).\nBelow, the bit structure for the IEEE 754 single precision format (32 bits), where the sign bit is the most significant (MSB), followed by the biased exponent and the fractional significand:\n31 30 23 22 0 ┌───┬──────────┬──────────────────────────┐ │ S │ Exponent │ Significand │ │ 1 │ 8 bits │ 23 bits │ └───┴──────────┴──────────────────────────┘ Special cases # IEEE 754 reserves specific patterns for edge situations:\nExponent Significand Meaning 00000000 000...0 Zero (\\(\\pm 0\\)) 00000000 \\(\\neq 0\\) Denormalized (subnormal) number 11111111 000...0 \\(\\pm\\infty\\) 11111111 \\(\\neq 0\\) NaN (Not a Number) The vacuum near zero and underflow If you look at the math of IEEE 754, you will notice there is a considerable \u0026ldquo;jump\u0026rdquo; in precision between the smallest possible normalized number and absolute zero. To keep calculations from abruptly losing precision and collapsing to zero when dealing with atomic-scale values (a problem known as underflow), the standard introduced denormalized numbers. They give up the implicit 1. in the significand to be able to represent absurdly small values (such as \\(1.4 \\times 10^{-45}\\) in single precision), filling this \u0026ldquo;vacuum\u0026rdquo; with a gradual degradation of precision.\nConverting to IEEE 754 single precision # \u0026#x2197; flowchart TD A[\"Signed decimal number\"] --\u003e B[\"1. Convert to binary\"] B --\u003e C[\"2. Normalize: 1.xxx × 2^E\"] C --\u003e D[\"3. Sign bit S\n0 = positive, 1 = negative\"] C --\u003e Exp[\"4. Biased exponent\nE_real + 127\"] C --\u003e Mant[\"5. Significand\nbits after the implicit 1\n(pad with zeros on the right)\"] D --\u003e R[\"Result: S | Exponent | Significand\"] Exp --\u003e R Mant --\u003e R Applying it to the example \\(-5.75\\):\nStep Operation Result 1. Convert to binary \\(5 = 101_2\\); \\(0.75 \\times 2 = 1.50\\) → 1; \\(0.5 \\times 2 = 1.0\\) → 1 \\(-101.11_2\\) 2. Normalize Move the point 2 places to the left \\(-1.0111 \\times 2^2\\) 3. Sign bit Negative 1 4. Biased exponent \\(E_{\\text{real}} = 2\\); \\(2 + 127 = 129\\) \\(10000001_2\\) 5. Significand Bits after 1.: 0111, pad with zeros 01110000000000000000000 Final result 1 10000001 01110000000000000000000 Check: $$(-1)^1 \\times 1.0111_2 \\times 2^{129-127} = -1.0111_2 \\times 2^2 = -101.11_2 = -5.75_{10} \\ \\checkmark$$ Example: decoding a custom 16-bit format A 16-bit format uses: 1 sign bit, 5 exponent bits with bias 15, and 10 significand bits (normalized, with implicit integer part = 1).\nThis format is not hypothetical: it is binary16, or half precision, from IEEE 754 itself, and it is one of the formats GPUs use to train and run machine learning models, where half the precision in exchange for half the memory and twice the bandwidth is a good deal.\nGiven the pattern BA32₁₆ = 1011 1010 0011 0010₂:\nSign: 1 → negative Exponent: 01110₂ = 14 → \\(E_{\\text{real}} = 14 - 15 = -1\\) Significand: 1000110010₂ Value: \\(-(1.1000110010_2) \\times 2^{-1}\\)\nConverting the significand: \\(1 + 2^{-1} + 2^{-5} + 2^{-6} + 2^{-9} = 1 + 0.5 + 0.03125 + 0.015625 + 0.001953\\ldots \\approx 1.5488\\)\nResult: \\(-1.5488 \\times 0.5 \\approx -0.7744\\)\nLargest positive normalized value: exponent = 11110₂ = 30 → \\(E = 15\\); significand = 1111111111₂ → \\(1.1111111111_2 \\times 2^{15} = 65504\\)\nSmallest positive normalized value: exponent = 00001₂ = 1 → \\(E = -14\\); significand = 0000000000₂ → \\(1.0 \\times 2^{-14} \\approx 0.000061\\)\nExample: decoding an IEEE 754 number Number with: sign = 0 (positive), significand = 1100100₂, real exponent = −3.\nNormalized number: \\(+(1.1100100)_2 \\times 2^{-3}\\)\nConverting the significand: \\(1 + 2^{-1} + 2^{-2} + 2^{-5} = 1 + 0.5 + 0.25 + 0.03125 = 1.78125\\)\nResult: \\(1.78125 \\times 2^{-3} = 1.78125 / 8 = \\mathbf{0.222656}\\)\nGoing deeper: the \u0026ldquo;From Zero to Float\u0026rdquo; series The IEEE 754 standard has many nuances beyond what we covered here: rounding, subnormal numbers, special operations, and the pitfalls of comparing floats. If you want to explore these topics in depth, including why 0.1 + 0.2 ≠ 0.3 in practically every programming language, the series From Zero to Float covers all of it with plenty of examples.\nPython bonus: inspecting bit representations # Python offers direct tools to inspect how numbers are stored in bits, using the struct module (for floating point) and the \u0026lt;\u0026lt; and \u0026gt;\u0026gt; operators (for integers).\nimport struct # IEEE 754 single precision representation of -5.75 raw = struct.pack(\u0026#34;\u0026gt;f\u0026#34;, -5.75) # \u0026#34;\u0026gt;f\u0026#34; = big-endian single float print(f\u0026#34;-5.75 as bytes : {raw.hex()}\u0026#34;) # c0b80000 bits = int.from_bytes(raw, \u0026#34;big\u0026#34;) print(f\u0026#34;In binary (32) : {bits:032b}\u0026#34;) # 11000000101110000000000000000000 # S=1 | Exp=10000001 | Mant=01110000000000000000000 ✓ # Unpacking back value = struct.unpack(\u0026#34;\u0026gt;f\u0026#34;, raw)[0] print(f\u0026#34;Back to float : {value}\u0026#34;) # -5.75 ✓ # Shifts and two\u0026#39;s complement with Python integers a = 0b00110010 # +50 in decimal b = 0b11010000 # in 8 bits unsigned = 208, in C2 = -48 print(f\u0026#34;\\na = {a} ({a:08b})\u0026#34;) print(f\u0026#34;a \u0026lt;\u0026lt; 1 = {a \u0026lt;\u0026lt; 1} ({a\u0026lt;\u0026lt;1:08b}) (× 2)\u0026#34;) print(f\u0026#34;a \u0026gt;\u0026gt; 1 = {a \u0026gt;\u0026gt; 1} ({a\u0026gt;\u0026gt;1:08b}) (÷ 2)\u0026#34;) # Two\u0026#39;s complement in Python (Python uses arbitrary-precision integers) def c2_negation(n: int, bits: int = 8) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;Return -n in two\u0026#39;s complement representation with \u0026#39;bits\u0026#39; bits.\u0026#34;\u0026#34;\u0026#34; mask = (1 \u0026lt;\u0026lt; bits) - 1 # 0xFF for 8 bits return (~n \u0026amp; mask) + 1 print(f\u0026#34;\\n-50 in C2 (8 bits) : {c2_negation(50):08b} = {c2_negation(50)}\u0026#34;) # 11001110 = 206 (unsigned) or -50 (C2) # Addition truncated to 8 bits def c2_add(a: int, b: int, bits: int = 8) -\u0026gt; int: mask = (1 \u0026lt;\u0026lt; bits) - 1 return (a + b) \u0026amp; mask print(f\u0026#34;50 + (-48) in 8 bits: {c2_add(a, b):08b} = {c2_add(a, b)}\u0026#34;) # 00000010 = 2 ✓ # Checking -5.75 as a double (64 bits) raw64 = struct.pack(\u0026#34;\u0026gt;d\u0026#34;, -5.75) bits64 = int.from_bytes(raw64, \u0026#34;big\u0026#34;) print(f\u0026#34;\\n-5.75 double (64 bits): {bits64:064b}\u0026#34;) # 1 10000000001 0111000000000000000000000000000000000000000000000000 Conclusion and next articles # Bits alone have no meaning: meaning emerges from the interpretation convention. ASCII turns 7 bits into a character; two\u0026rsquo;s complement turns a bit pattern into a negative integer; IEEE 754 turns 32 bits into a single-precision fractional number. Understanding these conventions is understanding how the computer works.\nIn the next article, we will explore the representation of machine instructions: how the bit patterns make up the instructions the CPU executes, the different instruction formats, and the addressing modes that determine where to fetch the operands. Understanding data representation is the fundamental step toward understanding instruction representation, since instructions are also just bit patterns, but with a specific structure that the hardware interprets to perform operations.\nSee you then!\n","date":"2026-04-28","externalUrl":null,"permalink":"/2026/04/data-representation/","section":"Blog","summary":"Uppercase and lowercase differ by a single bit in the ASCII table, and that was no accident. To the hardware everything is bits, and the value depends on the reading convention: ASCII, Unicode, and UTF-8 for text, two’s complement for integers, and IEEE 754 for real numbers.","title":"Data Representation: Bits That Mean Something","type":"posts"},{"content":"","date":"2026-04-28","externalUrl":null,"permalink":"/tags/ieee-754/","section":"tags","summary":"","title":"IEEE 754","type":"tags"},{"content":"","date":"2026-04-28","externalUrl":null,"permalink":"/tags/unicode/","section":"tags","summary":"","title":"Unicode","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/barramentos/","section":"tags","summary":"","title":"barramentos","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/buses/","section":"tags","summary":"","title":"buses","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/cpu/","section":"tags","summary":"","title":"CPU","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/desempenho/","section":"tags","summary":"","title":"desempenho","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/performance/","section":"tags","summary":"","title":"performance","type":"tags"},{"content":"","date":"2026-04-23","externalUrl":null,"permalink":"/tags/pipeline/","section":"tags","summary":"","title":"pipeline","type":"tags"},{"content":" In the previous article, CPU in Action: The Hypothetical Processor, we followed our 12-bit processor executing one instruction at a time: fetch from memory, decode, execute, write the result, and start over. The sequence is correct and complete, but it carries a fundamental inefficiency: while the ALU executes an instruction, the memory bus sits idle; while a new fetch happens, the ALU waits. It is as if an automobile assembly line never touched the next car before the previous one was completely finished.\nThe pipeline technique eliminates exactly this waste: just as an industrial assembly line keeps several cars at different stages at the same time, the pipeline keeps several instructions at different stages of the cycle simultaneously. The result is a significant increase in throughput without any individual instruction becoming faster, all thanks to overlapping operations.\nIn this article you will understand:\nthe assembly line analogy and the performance formulas; the classic 2-stage and 5-stage pipelines; the three types of hazards that interrupt the pipeline flow; how to calculate the real speedup with numerical examples; the hierarchy of buses that connects CPU, memory, and peripherals; the differences between synchronous and asynchronous buses; the two ways to implement the control unit: hardwired and microprogrammed. The assembly line analogy # Imagine a factory that builds cars in four independent steps (frame, engine, paint, and finishing), each taking exactly one week. Without a pipeline, the second car only enters the line when the first is completely finished: four cars take 16 weeks. With a pipeline, each step receives a different car every week: after the initial four weeks of \u0026ldquo;filling\u0026rdquo; the line, one finished car comes out every week.\nSituation 1 car 4 cars Without pipeline 4 weeks 16 weeks With pipeline 4 weeks 7 weeks The pipeline technique works like an assembly line: different instructions occupy different stages at the same time. The gain does not come from making an individual instruction faster, but from increasing the number of instructions completed per unit of time (throughput).\nIn an idealized model, in which:\nall stages take exactly 1 cycle; there are no hazards; there are no resource conflicts; the pipeline is always kept fed, we have:\n$$ T_{\\text{without pipeline}} = N \\times K $$$$ T_{\\text{with pipeline}} = K + (N - 1) $$where:\n\\(N\\) = number of instructions; \\(K\\) = number of pipeline stages. The term \\(K\\) represents the initial filling time of the line. After that, in the ideal regime, one instruction finishes per cycle. The term \\((N-1)\\) is the additional cycles after the first stage is full. For large programs, where \\(N \\gg K\\), the time tends to \\(N\\) cycles, that is, one instruction finished per cycle.\nThese formulas hold for the ideal case In real processors, data, control, and structural hazards can insert stalls and flushes, reducing the effective gain of the pipeline.\nThe gain grows with N The fixed overhead of \\(K\\) cycles to fill the pipeline is diluted as more instructions are processed. For a program with millions of instructions, the speedup approaches \\(K\\) times. For few instructions, the startup cost can make the pipeline less advantageous than sequential execution.\nTo understand this, let us recall the traditional instruction cycle seen in earlier articles, which is divided into steps such as fetch, decode, execute, and write result. The pipeline divides this cycle into stages that can be processed simultaneously by different instructions. The figure below recovers this cycle with the steps named, to serve as a reference for what comes next:\nInstruction cycle In sequential execution, each instruction goes through all the steps before the next one starts, as illustrated below:\nExample of sequential execution Note that, assuming each stage takes the same time T, executing the 4 instructions sequentially would take 8T (4 instructions × 2 stages each).\nIn the next sections, we will explore how the pipeline stages are organized and what challenges arise when multiple instructions compete for shared resources.\nThe pipeline stages # Two-stage pipeline # The simplest division separates the cycle into fetch (Fetch) and execution (Execute). While the ALU executes instruction \\(i\\), the CU already fetches instruction \\(i+1\\) from memory: the two stages work in parallel.\nStage Function Fetch Fetches the next instruction from memory via MAR/MDR Execute Decodes the instruction and performs the operation The figure below illustrates what happens with the same 4-instruction program seen earlier, but now with the 2-stage pipeline. Note that from the second cycle on, the instructions overlap: while the first is executing, the second is already being fetched; when the second is executing, the third is already being fetched; and so on:\n2-stage pipeline In the figure, the stages appear abbreviated as L (leitura, \u0026ldquo;read\u0026rdquo;, the same as the Fetch stage) and E (execução, execution).\nThanks to the overlap, the total time for the 4 instructions drops from 8T to 5T, a gain of 1.6×.\nThe problem arises with conditional branch instructions (such as JZ or JP in our hypothetical processor): when the condition is tested in the Execute stage, the instruction already fetched in the Fetch stage may be the wrong one. It must be discarded (flush) and the instruction at the correct branch address must be fetched.\nThe solution is branch prediction. The simplest form of it is static \u0026ldquo;not-taken\u0026rdquo; prediction: we assume the branch will not occur and keep fetching instructions sequentially. If the prediction fails, we flush and restart the fetch. Correct predictions eliminate the penalty; wrong predictions impose, in this two-stage model, only a one-cycle delay.\nModern processors do not stop there. They use dynamic predictors, which learn the behavior of each branch during execution: 2-bit saturating counters (which only change their mind after being wrong twice in a row, which gets loops right almost always), two-level predictors correlating the history of several branches, and more elaborate schemes such as TAGE and perceptron-based predictors. The hit rate exceeds 95% on real code, and that is exactly why a miss hurts so much in a deep pipeline, which already has dozens of instructions in flight when it discovers it bet on the wrong path.\nWhen speculation became a vulnerability Getting work done ahead of time based on a guess has a consequence that only became clear in 2018, with Spectre. When the processor speculatively executes instructions from a path that will end up discarded, it undoes the effect on the program\u0026rsquo;s visible state (registers, memory), but it does not undo the effect on the cache. The blocks loaded during that discarded work are still there.\nThis opens a door: a malicious program can induce the processor to speculate on an access it would not be allowed to make and then discover what was read just by measuring cache access times. There is no implementation bug: branch prediction worked as designed. The attack exploits the difference between what the processor undoes and what it leaves as a trace, and it forced the entire industry to revise, at a real performance cost, a technique that dated from the 1980s.\nBranch penalty In very simple models, the penalty can be just 1 cycle. In deeper pipelines, the penalty can be larger, because more instructions will already have been partially processed before the branch is resolved.\nFive-stage pipeline # RISC architectures refined the pipeline to five stages, making each step more granular, and therefore faster, since each stage requires less logic. The classic RISC pipeline uses the stages IF, ID, EX, MEM, and WB (fetch, decode, execute, memory access, and write back). Here we adopt a similar didactic division, which separates the address calculation from the operand fetch:\n\u0026#x2197; flowchart LR B[\"F\nFetch\"] --\u003e D[\"D\nDecode\"] --\u003e C[\"C\nAddress\nCalculation\"] --\u003e O[\"O\nOperand\nFetch\"] --\u003e E[\"E\nExecute\"] Stage Abbreviation Function Fetch F Fetches the instruction from main memory (MM) via PC Decode D Decodes the opcode; identifies operands in registers Address calculation C Computes the effective address of the operand in memory Operand fetch O Fetches the operand from MM or from a register Execute E Performs the operation in the ALU and/or stores the result With five stages, the pipeline can keep five instructions in progress simultaneously: each one in a different stage. The clock frequency can be increased because each stage is simpler; and the throughput, in steady state, approaches one instruction completed per cycle.\nThe figure below shows the same 4-instruction program, now with the 5-stage pipeline. In the ideal case, from the fifth cycle on all five steps would be busy and one instruction would finish every cycle. That is not what happens here, and the figure shows why: the blue blocks are the stages that access memory, the hatched areas are waits, and the bottom strip indicates which instruction memory is serving in each cycle.\nFour instructions in a five-stage pipeline in which F, O, and E compete for the same memory. In each cycle the memory goes to the oldest instruction that needs it, and the waits bring the total to 14T. If the pipeline were sequential, the 4 instructions would take 20T (4 instructions × 5 stages each). With the 5-stage pipeline, the total time drops to 14T, a gain of approximately 1.43×.\nBut 14T is not what the ideal-case formula predicts. With \\(K = 5\\) and \\(N = 4\\), \\(K + (N-1)\\) would give 8T. The difference of six cycles has a concrete cause, and it is the first appearance of a hazard in this article: only one memory access can be performed at a time. And three of the five stages need it: F fetches the instruction, O fetches the operand, and E writes the result. When two of them want the memory in the same cycle, one waits. This is what the next section calls a structural hazard, and it is why the figure shows waits where the ideal model would show a continuous diagonal.\nYou can follow each wait in the bottom strip. Between 4 and 5, instruction 2 could already fetch its operand, but memory is with E1, which writes the result of instruction 1. Instruction 3 waits two cycles for the same reason, now with O2 and E2. And instruction 4, whose fetch could start at 3, only gets memory at 9, because until then it is always served last.\nAbout the scheduling in the figure The figure follows a simple rule: in each cycle, memory goes to the oldest instruction that needs it. It is a reasonable rule (what started first finishes first), but it is not the best possible. Starting the fetch of the fourth instruction at 7, before the third finishes, the total would drop to 12T. With a different priority policy, the total changes. What is worth taking away is the existence of the conflict and the order of magnitude of what it costs, not the exact value of 14T, which depends on the rule adopted.\nAnd there is a number that bothers us: the 5-stage pipeline yielded 1.43×, less than the 1.6× of the 2-stage pipeline. Do more stages make performance worse? No. There are two combined effects, and neither is the fault of depth itself:\nThe memory conflict we just saw. Without it, the 4 instructions would finish in 8T, which would give 2.5×, well above the 2-stage pipeline. The short program. A pipeline with \\(K\\) stages spends \\(K - 1\\) cycles before completing the first instruction, and this fixed cost is only diluted with many instructions. With \\(N = 4\\), the filling is almost the entire program. The speedup calculation section, further ahead, shows the same 5-stage pipeline reaching 3.27× with 100 instructions, against 1.68× for the 2-stage one. The hierarchy flips as soon as the program has a realistic size, and the trend, for \\(N \\gg K\\), is for the gain to approach \\(K\\).\nHazards: when the pipeline stumbles # The ideal pipeline assumes that each stage finishes in the same time and that instructions are independent of each other. In practice, three situations break this assumption, the so-called hazards:\nHazards reduce real performance When a hazard is detected, the pipeline may need stalls (bubbles, that is, cycles wasted waiting) or flushes (discarding instructions already fetched). Real performance falls below the theoretical ideal.\nType Cause Solution Structural Two stages need the same hardware resource at the same time (e.g., a single memory bank for fetch and for data) Duplicate the resource (separate instruction and data caches); insert a stall Data An instruction depends on the result of a previous instruction that has not yet finished executing Forwarding (passing the result directly between stages); stall Control A conditional branch instruction invalidates instructions already fetched Branch prediction; flush of the incorrect instructions These hazards are the main reason why the real performance of the pipeline falls below the theoretical limit.\nMost optimizations in modern compilers (such as instruction reordering and loop unrolling) aim precisely at reducing the frequency of data and control hazards.\nCalculating performance # Even with the delays caused by hazards, the mathematics of the pipeline still proves its superiority over sequential execution at large scale. To consolidate this concept, the following examples illustrate how to apply the formulas in practice. The most important result is not the final number itself, but the intuition about when it is worth using this technique (or not).\nExample: four-stage pipeline Consider an instruction divided into 6 blocks with times: $$ A=80,\\ B=20,\\ C=70,\\ D=40,\\ E=60,\\ F=10\\ \\text{ps} $$The execution time of an instruction without a pipeline is the sum of all the blocks: $$ T_{\\text{instr}} = 80+20+70+40+60+10 = 280\\ \\text{ps} $$For 10 instructions executed sequentially, the total time will be: $$ T_{\\text{seq}} = 10 \\times 280 = 2800\\ \\text{ps} $$Now, to implement a 4-stage pipeline, we group the blocks so as to try to balance the times:\nStage 1: \\(A = 80\\) ps Stage 2: \\(B + C = 20 + 70 = 90\\) ps Stage 3: \\(D = 40\\) ps Stage 4: \\(E + F = 60 + 10 = 70\\) ps The pipeline cycle is determined by the slowest stage: $$ T_{\\text{cycle}} = \\max(80,90,40,70) = 90\\ \\text{ps} $$For 10 instructions, the total time with the pipeline will be: $$ T_{\\text{pipeline}} = 4 \\times 90 + (10-1)\\times 90 $$ $$ T_{\\text{pipeline}} = 360 + 810 = 1170\\ \\text{ps} $$The speedup is: $$ \\text{Speedup} = \\frac{T_{\\text{seq}}}{T_{\\text{pipeline}}} = \\frac{2800}{1170} \\approx 2.39 $$Therefore:\ntime without pipeline: \\(2800\\ \\text{ps}\\) time with pipeline: \\(1170\\ \\text{ps}\\) speedup: \\(\\approx 2.39\\times\\) The central point is this: the performance of the pipeline depends heavily on the balance between the stages. The slowest stage sets the pace of the whole line, so the machine must be designed with all stages running at the time of the slowest one: in this case, 90 ps.\nExample: pipeline with real instructions (2 and 5 stages) A machine divides each instruction into five blocks with the following durations: Fetch (IF = 10 ps), Decode (ID = 2 ps), Address calculation (AC = 2 ps), Operand fetch (OF = 10 ps), and Execute (EX = 10 ps). Calculate the time to execute 100 instructions in the three configurations.\na) Sequential: $$T_{\\text{instr}} = 10+2+2+10+10 = 34\\ \\text{ps} \\quad\\Rightarrow\\quad T_{\\text{total}} = 100 \\times 34 = \\mathbf{3400\\ \\text{ps}}$$b) 2-stage pipeline (S1 = {IF, ID, AC} = 14 ps; S2 = {OF, EX} = 20 ps): $$T_{\\text{cycle}} = \\max(14, 20) = 20\\ \\text{ps}$$ $$T_{\\text{total}} = \\bigl[2 + (100-1)\\bigr] \\times 20 = 101 \\times 20 = \\mathbf{2020\\ \\text{ps}}$$c) 5-stage pipeline (each original block is a stage): $$T_{\\text{cycle}} = \\max(10, 2, 2, 10, 10) = 10\\ \\text{ps}$$ $$T_{\\text{total}} = \\bigl[5 + (100-1)\\bigr] \\times 10 = 104 \\times 10 = \\mathbf{1040\\ \\text{ps}}$$ Configuration Total time Speedup Sequential 3400 ps 1× 2-stage pipeline 2020 ps 1.68× 5-stage pipeline 1040 ps 3.27× The 5-stage pipeline offers a speedup of 3.27× with the same hardware blocks, just by reorganizing the execution flow.\nThe two previous examples use the ideal model: no overhead beyond the initial filling of the pipeline. The following example introduces an extra startup cost, representing real situations where the pipeline needs to be \u0026ldquo;warmed up\u0026rdquo; by preparatory instructions before reaching steady state. This overhead makes the break-even point nontrivial and more instructive.\nExample: when is it worth using the pipeline? Consider again the machine with a 2.5 GHz clock: $$T_{\\text{cycle}} = \\frac{1}{2.5 \\times 10^9} = 0.4\\ \\text{ns per cycle}$$Each instruction takes 2 cycles in sequential execution. For 1250 instructions: $$T_{\\text{seq}} = 1250 \\times 2 \\times 0.4 = \\mathbf{1000\\ \\text{ns}}$$2-stage pipeline with 2 initialization instructions (each = 2 stages): $$T_{\\text{init}} = 2 \\times 2 \\times 0.4 = 1.6\\ \\text{ns}$$ $$T_{\\text{pipeline}}(N) = 1.6 + 2 \\times 0.4 + (N-1) \\times 0.4 = 0.4N + 2.0\\ \\text{ns}$$For the pipeline to pay off: \\(0.4N + 2.0 \\leq 0.8N\\), that is, \\(N \\geq 5\\) instructions.\n3-stage pipeline with 2 initialization instructions. Note the change of reference: dividing the instruction into three stages means the equivalent sequential machine spends 3 cycles per instruction, not 2, so the fair comparison is now against \\(3 \\times 0.4N = 1.2N\\). $$T_{\\text{pipeline}}(N) = 2 \\times 3 \\times 0.4 + 3 \\times 0.4 + (N-1) \\times 0.4 = 0.4N + 3.2\\ \\text{ns}$$For it to pay off: \\(0.4N + 3.2 \\leq 1.2N\\), that is, \\(N \\geq 4\\) instructions.\nNotice that the 3-stage pipeline pays off with fewer instructions (4) than the 2-stage one (5), which seems contradictory at first glance. The reason is that the baseline against which it is measured also became slower, for having more stages. Comparing pipelines of different depths requires attention to what each one is being measured against.\nThe practical conclusion: pipelines with very few instructions can be slower than sequential execution, due to the startup overhead. For real programs with thousands of instructions, the pipeline is always advantageous.\nBuses: the backbone of communication # While the pipeline speeds up the execution of instructions inside the CPU, buses solve a different problem: how CPU, memory, and peripherals exchange data with each other. A bus is simply a set of shared electrical lines: a collective communication channel.\nThere are three functional types of bus, each carrying one kind of information:\nBus Function Example signals Data (DB) Transfers data between components 8-, 16-, 32-, or 64-bit words Address (AB) Carries addresses issued by the CPU Address of the cell to read or write Control (CB) Carries timing and control signals READ, WRITE, CLOCK, RESET, INTERRUPT The width of the address bus determines the theoretical addressing space:\n$$ \\text{Possible addresses} = 2^N $$where \\(N\\) is the number of bits of the AB.\nFor example, an address bus with 32 bits can represent \\(2^{32}\\) distinct addresses. In byte-addressable systems, this corresponds to a theoretical space of \\(2^{32}\\) bytes.\nBus hierarchy # A real computer does not have a single global bus, which would create an enormous bottleneck.\nHistorically, personal computers used a hierarchy of buses with elements such as FSB, Northbridge, Southbridge, AGP, and ISA, with different speeds suited to each component:\n\u0026#x2197; flowchart TD UCP[\"CPU\n(Processor)\"] \u003c--\u003e BL[\"Local Bus / FSB\n(Front-Side Bus)\"] BL \u003c--\u003e CacheL2[\"L2 Cache\"] BL \u003c--\u003e PN[\"Northbridge\"] PN \u003c--\u003e MP[\"Main Memory\"] PN \u003c--\u003e BAD[\"High-Performance Bus\n(AGP / PCIe)\"] BAD \u003c--\u003e GPU[\"Graphics Card / GPU\"] PN \u003c--\u003e PS[\"Southbridge\"] PS \u003c--\u003e BExp[\"Expansion Bus\n(PCI / ISA / USB)\"] BExp \u003c--\u003e Per1[\"Disks / Storage\"] BExp \u003c--\u003e Per2[\"USB Peripherals\"] The Northbridge manages the high-speed traffic between CPU, RAM, and graphics card. The Southbridge takes care of the slower peripherals (disks, USB, audio). In modern architectures, much of this logic has been absorbed into the processor itself, but the hierarchical concept remains.\nThis organization is an excellent didactic reference, but it no longer represents the main way modern platforms are organized.\nFor a present-day reader, it is worth presenting this in two layers:\nHistorical view # CPU connected to the chipset by the front-side bus (Front-Side Bus); Northbridge handling RAM and video; Southbridge handling USB, disks, and slow peripherals. Modern view # the memory controller is usually integrated into the processor itself; communication with high-speed devices happens through interconnects such as PCI Express; the link to the chipset/PCH happens through dedicated links; the hierarchy still exists, but with fewer explicit \u0026ldquo;bridges\u0026rdquo; visible to the user. History and design characteristics # For this hierarchical architecture to work without creating traffic bottlenecks, the communication paths had to widen and speed up over time. The evolution of expansion bus standards over the decades reveals exactly this continuous search for greater bandwidth:\nBus Year Width Clock Transfer Rate ISA (original PC) 1981 8 bits n/a ~1 MB/s ISA (AT) 1984 16 bits 8 MHz ~8 MB/s PCI 1992 32 bits 33 MHz 133 MB/s PCI (extended) 1995 64 bits 66 MHz 533 MB/s AGP 1× 1997 32 bits 66 MHz 266 MB/s AGP 8× 2002 32 bits 533 MHz effective 2,133 MB/s USB 1.0 1996 Serial n/a 1.5 / 12 Mbit/s USB 2.0 2000 Serial n/a 480 Mbit/s With the years in the table, the arc becomes visible: over two decades, the transfer rate of the expansion bus went from about 1 MB/s to more than 2 GB/s, a factor of two thousand. And the table could continue with the most recent standards: PCI Express, which debuts in 2003 and reaches versions 4.0, 5.0, and 6.0 with bandwidths in the range of GB/s per lane, combining multiple lanes to reach even higher rates.\nFour design parameters determine the performance of a bus:\nWidth: number of data lines (more lines allow transferring more bits per cycle); Protocol: defines the timing of each signal and the handshake between devices; Speed: bus clock frequency; Transfer modes: one block per cycle, or multiple consecutive blocks (burst mode). The bus cycle: who speaks and when # Any transfer on a bus involves two roles and an arbiter:\nRole Who takes it Function Master CPU, DMA controller Starts the transfer Slave Main memory, I/O device Responds to the request Arbiter Dedicated circuit Decides who uses the bus when there is a conflict The basic sequence is:\n$$ \\text{request} \\to \\text{grant} \\to \\text{use} \\to \\text{release} $$The CPU\u0026rsquo;s Control Unit takes part in generating the internal and external signals needed for operation, but bus arbitration should be treated as a function of the interconnect subsystem\u0026rsquo;s arbitration mechanism, and not as the exclusive responsibility of the CU.\nDirect Memory Access (DMA) As we have seen, the CPU is the natural master of the system. However, if the CPU had to act as an intermediary for every byte read from a large file on the hard disk into RAM, it would waste millions of cycles just moving data, without processing anything.\nThe hardware solution for this is DMA (Direct Memory Access). It allows peripheral controllers (such as that of the hard disk or the graphics card) to temporarily take on the role of bus Master. With the arbiter\u0026rsquo;s permission, the disk writes the file directly into Main Memory, leaving the CPU completely free to execute other instructions in parallel.\nSynchronous × asynchronous # The central difference between synchronous and asynchronous buses is not simply \u0026ldquo;one being faster than the other\u0026rdquo;, but how the timing coordination is done.\nCharacteristic Synchronous Asynchronous Coordination Based on a common clock Based on request and acknowledge signals Implementation Simpler to test and integrate More flexible for heterogeneous components Adaptation to slow devices More limited Better adaptation Typical use Fast, nearby components Devices with varying timings In summary:\nthe synchronous bus is simpler when the components operate at a similar pace; the asynchronous bus is more appropriate when there is great speed variation among the devices. Handshake protocol (asynchronous bus) On the asynchronous bus, the master sends a request (REQ) signal and waits for the slave\u0026rsquo;s acknowledgment (ACK) before proceeding. Only after receiving the ACK is the next operation started. This three-step protocol allows devices with very different speeds to be integrated on the same bus: the bus adapts to the pace of the slowest component, without requiring all of them to share a common clock.\nThe Control Unit from the inside # With the assembly line of instructions (pipeline) running at high speed and the communication paths (buses) established, one last question remains: who manages this whole orchestra? The answer takes us back to the heart of the CPU: the Control Unit (CU). It is the one that fetches each instruction, decodes the opcode, and emits the electrical signals that activate each component at the right time, including bus arbitration.\nAlthough we have focused on fetch and execution so far, in practice the CU divides the processor\u0026rsquo;s time into four fundamental subcycles:\n1 Fetch Goes to memory and brings the instruction. 2 Indirect If the instruction uses a pointer (indirect addressing), the CU must do an extra memory read to discover the real address of the data. 3 Execute Triggers the ALU or moves the data as decoded. 4 Interrupt The CU pauses what it is doing to handle an urgent external event (such as a network packet arriving or a hardware error) before fetching the next instruction. But how does the CU itself \u0026ldquo;know\u0026rdquo; how to do this? It must be built in some physical form, and there are two classic approaches for that.\nHardwired implementation # The hardwired CU is built entirely in fixed hardware: logic gates, flip-flops, and dedicated combinational circuits. Each instruction of the instruction set corresponds to a specific logical path in the circuit. The result is an extremely fast CU, since the control logic runs directly in hardware with no memory lookup.\nFor those familiar with programming, the best way to picture a hardwired CU is to imagine a gigantic set of IF/AND/OR conditionals physically converted into wires and transistors.\nFor example, if the CPU needs to activate the control signal \\(C_5\\) (which opens a register\u0026rsquo;s gate) only during time cycle \\(t_2\\) of an addition instruction (ADD), the engineers create a logic circuit that solves the Boolean equation: \\(C_5 = \\text{ADD} \\text{ AND } t_2\\). When electricity flows through that specific combination, the gate opens. It is pure mathematics governing silicon.\nThe drawback is rigidity: any change to the instruction set requires redesigning the physical circuit. That is why the hardwired implementation is the natural choice for RISC architectures, whose instruction set is small, regular, and stable.\nMicroprogrammed implementation # In the microprogrammed implementation, the Control Unit executes microinstructions stored in an internal control memory. Instead of relying only on fixed combinational circuits, the behavior of each instruction is described by an organized sequence of microoperations.\nThis makes the CU design more flexible and easier to structure than a fully hardwired control, although usually at a lower speed. Microprogramming makes it easier to implement and evolve the control logic, especially in architectures with more complex instructions.\nMicroinstructions can be organized in two ways:\nType Word width Parallelism Encoding Horizontal Long (many bits) High: several micro-ops at once Minimal: almost one bit per control signal Vertical Short and compact Low: few active fields at a time High: needs an additional decoder Overall comparison # Characteristic Hardwired Microprogrammed Speed Very fast Moderate Flexibility Low High Cost of change High (hardware redesign) Low (firmware update) Design complexity High Moderate Typical architecture RISC CISC Python bonus: calculating pipeline performance # The pipeline formulas are simple enough to become reusable Python functions. With them, you can quickly explore how different groupings of stages affect the total time:\ndef sequential_time(stage_durations: list[float], n_instructions: int) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Total execution time without pipeline (all stages in series).\u0026#34;\u0026#34;\u0026#34; return sum(stage_durations) * n_instructions def pipeline_time(stage_durations: list[float], n_instructions: int) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Total time with pipeline. The cycle width is set by the slowest stage (bottleneck). \u0026#34;\u0026#34;\u0026#34; k = len(stage_durations) cycle_width = max(stage_durations) cycles = k + (n_instructions - 1) return cycles * cycle_width def speedup(t_seq: float, t_pip: float) -\u0026gt; float: return t_seq / t_pip # Example: 6 tasks with durations 3, 1, 2, 3, 2, 2 ns grouped into 4 stages stages = [3, 3, 3, 4] # sum per stage after grouping n = 6 t_s = sequential_time([3, 1, 2, 3, 2, 2], n) t_p = pipeline_time(stages, n) print(f\u0026#34;Sequential : {t_s:.0f} ns\u0026#34;) print(f\u0026#34;Pipeline : {t_p:.0f} ns\u0026#34;) print(f\u0026#34;Speedup : {speedup(t_s, t_p):.2f}×\u0026#34;) # Sequential : 78 ns # Pipeline : 36 ns # Speedup : 2.17× # Find the minimum number of instructions for the pipeline to pay off # (with startup cost) def min_pipeline_instructions( pipeline_cycle_time: float, seq_cycles_per_instr: int, base_cycle_time: float, n_init_instructions: int, init_stages: int, ) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;Returns the minimum N such that the pipeline is faster than sequential.\u0026#34;\u0026#34;\u0026#34; t_init = n_init_instructions * init_stages * base_cycle_time for n in range(1, 10_000): t_pip = t_init + init_stages * pipeline_cycle_time + (n - 1) * pipeline_cycle_time t_seq = n * seq_cycles_per_instr * base_cycle_time if t_pip \u0026lt;= t_seq: return n return -1 # does not converge in the tested range # Machine at 2.5 GHz: cycle time = 0.4 ns t_base = 0.4 # ns print(f\u0026#34;Min. instrs (K=2): {min_pipeline_instructions(t_base, 2, t_base, 2, 2)}\u0026#34;) print(f\u0026#34;Min. instrs (K=3): {min_pipeline_instructions(t_base, 3, t_base, 2, 3)}\u0026#34;) # Min. instrs (K=2): 5 # Min. instrs (K=3): 4 Conclusion and next articles # The pipeline turns sequential execution into an assembly line of instructions, multiplying the throughput by the number of stages, with the caveat that data, control, and structural hazards can introduce stalls that reduce the real gain. Buses, in turn, form the communication infrastructure among all components, with synchronous protocols for fast components and asynchronous ones for peripherals of varying speeds. And the choice between hardwired and microprogrammed CU reflects the tradeoff between speed and flexibility.\nIn the next article, we will study data representation: how integers, text, and real numbers are encoded in bits. We will explore the most common encoding standards (ASCII, Unicode, two\u0026rsquo;s complement, and the IEEE 754 standard) and understand the implications of each choice for data processing.\nSee you then!\n","date":"2026-04-23","externalUrl":null,"permalink":"/2026/04/pipeline-and-buses/","section":"Blog","summary":"An assembly line does not wait for one car to be finished before starting the next, and the pipeline does the same with instructions. We will see the stages, the hazards that stall the line, the speedup calculation, the buses that connect CPU, memory, and peripherals, and the control unit.","title":"Pipeline and Buses: Parallelism and Communication in the CPU","type":"posts"},{"content":"","date":"2026-04-21","externalUrl":null,"permalink":"/tags/ciclo-de-instrucao/","section":"tags","summary":"","title":"ciclo de instrução","type":"tags"},{"content":" In the previous article, CPU: Components and Instruction Cycle, we introduced the blocks that make up the CPU: ALU, CU, registers, decoder, and clock. We described the microevents of the fetch cycle in RTL notation and saw how the PC, the MAR, and the IR cooperate to bring an instruction from memory. But we described everything in general terms. Now let\u0026rsquo;s get concrete: we will follow a hypothetical processor executing real instructions, bit by bit.\nUsing a simplified processor as a teaching model is a classic strategy in the field: we eliminate the complexity of commercial architectures (x86, ARM) and focus on the fundamental mechanisms that any real processor shares.\nIn this article you will learn:\nthe organization of the hypothetical 12-bit processor; the complete instruction set and its RTL notation; how to encode an instruction in binary and hexadecimal; the flowchart of the complete instruction cycle; the step-by-step execution of LDA and ADD with state tables; the fundamentals of assembly language; complete assembly programs for practical problems. The hypothetical processor # Despite all the physical complexity of modern chips, the logical foundations that operate inside them remain surprisingly similar to those of their ancestors. To understand how the CPU fetches and executes instructions without the \u0026ldquo;pollution\u0026rdquo; of billions of transistors and complex pipelines, let\u0026rsquo;s take a step back. We will use as a base a hypothetical computer model of just 12 bits.\nThe hypothetical processor has a lean organization that keeps all the essential components of a real processor. Its complete specification:\nComponent Size Rationale Word 12 bits Size of every memory cell Addresses 8 bits \\(2^8 = 256\\) addressable cells Memory cells 12 bits One instruction or datum per cell ACC (Accumulator) 12 bits Same size as the word PC 8 bits Enough to address the 256 cells MAR 8 bits Same size as the addresses MDR / MBR 12 bits Same size as the word IR 12 bits Stores a complete instruction Notice the coherence: the PC has 8 bits because 8 bits suffice to address 256 cells; the MAR also has 8 bits because it needs to carry addresses; the MDR, the IR, and the ACC have 12 bits because they work with complete words.\nInstruction format # Each instruction occupies exactly one memory cell, with 12 bits divided into two fields:\n$$\\underbrace{b_{11}\\ b_{10}\\ b_9\\ b_8}_{\\text{Opcode (4 bits)}} \\quad \\underbrace{b_7\\ b_6\\ b_5\\ b_4\\ b_3\\ b_2\\ b_1\\ b_0}_{\\text{Operand (8 bits)}}$$ 4 opcode bits → \\(2^4 = 16\\) distinct possible operations 8 operand bits → \\(2^8 = 256\\) addresses, covering the entire memory The figure below shows this same division with a real instruction filled in: 1B4, which appears later in the article. Two things are worth noticing: each hexadecimal digit corresponds to exactly four bits, which is why the opcode takes up the entire first digit; and the operand, with its eight bits, goes from 00 to FF, the limit that every address of this processor must respect.\nThe 12 bits of instruction 1B4: the first 4 form the opcode (0001, LDA) and the next 8 the operand (1011 0100, address B4). The number of possible operations and the addressing space grow exponentially with the available bits. Adding 1 bit to the opcode doubles the number of instructions; adding 1 bit to the operand doubles the addressable memory space.\nExample: sizing an instruction set A processor has 128 distinct instructions and a memory with 512 words. How many bits are needed for the opcode and for the operand? What is the total size of the instruction?\nOpcode: \\(\\log_2 128 = \\log_2 2^7 = \\mathbf{7\\ \\text{bits}}\\)\nOperand: \\(\\log_2 512 = \\log_2 2^9 = \\mathbf{9\\ \\text{bits}}\\)\nInstruction = \\(7 + 9 = \\mathbf{16\\ \\text{bits}}\\)\nRegisters: PC and MAR → \\(\\mathbf{9}\\) bits (they address memory); MDR and IR → \\(\\mathbf{16}\\) bits (they store words). Total capacity: \\(512 \\times 16 = 8192\\ \\text{bits} = \u003e 1\\ \\text{Kbyte}\\).\nEffect of adding 1 bit to the opcode If this processor went from 16 to 17 bits per instruction (keeping the operand at 9 bits), the opcode would have \\(17 - 9 = 8\\) bits → \\(\\mathbf{256}\\) possible operations. A single extra bit doubles the instruction repertoire.\nThe instruction set # The instruction set of the hypothetical processor has 15 entries, covering arithmetic operations, data movement, conditional and unconditional branches, and I/O:\nOpcode (hex) Mnemonic Operation in RTL Description 0 HLT (none) Stop execution 1 LDA Op \\(\\text{ACC} \\leftarrow (Op)\\) Load from memory into the ACC 2 STR Op \\((Op) \\leftarrow \\text{ACC}\\) Store the ACC in memory 3 ADD Op \\(\\text{ACC} \\leftarrow \\text{ACC} + (Op)\\) Add 4 SUB Op \\(\\text{ACC} \\leftarrow \\text{ACC} - (Op)\\) Subtract 5 JZ Op If \\(\\text{ACC} = 0\\): \\(\\text{PC} \\leftarrow Op\\) Branch if zero 6 JP Op If \\(\\text{ACC} \u003e 0\\): \\(\\text{PC} \\leftarrow Op\\) Branch if positive 7 JN Op If \\(\\text{ACC} \u003c 0\\): \\(\\text{PC} \\leftarrow Op\\) Branch if negative 8 JMP Op \\(\\text{PC} \\leftarrow Op\\) Unconditional branch 9 GET Op Input \\(\\rightarrow (Op)\\) Read from the input port A PRT Op \\((Op) \\rightarrow\\) output Print/send to output B MPY Op \\(\\text{ACC} \\leftarrow \\text{ACC} \\times (Op)\\) Multiply C DIV Op \\(\\text{ACC} \\leftarrow \\text{ACC} \\div (Op)\\) Divide D INC \\(\\text{ACC} \\leftarrow \\text{ACC} + 1\\) Increment the ACC E DCR \\(\\text{ACC} \\leftarrow \\text{ACC} - 1\\) Decrement the ACC The code F remains free for future extensions or reserved.\nThis processor has relatives, and one of them runs in the browser The machine in this article is not an isolated invention. Practically every architecture course builds a similar one, and several of them have a name, a book, and an implementation. The best known are Neander and Ramses, defined by Raul and Taisy Weber, from UFRGS, described in Fundamentos de Arquitetura de Computadores and used in dozens of Brazilian courses; MARIE, by Null and Lobur; MIC-1, by Tanenbaum; SAP-1, by Malvino; and the Little Man Computer, proposed by Stuart Madnick in 1965. The processor presented here is, in essence, a Neander with more instructions: a single accumulator, one instruction per word, conditional branches that test the ACC.\nKnowing this has a practical advantage: you can run the programs instead of merely following them on paper. The Little Man Computer has a web simulator that works in the browser, without installing anything: you type the program, run it step by step, and watch the accumulator and the program counter change at each cycle. Its repertoire is smaller than this article\u0026rsquo;s, but it translates directly: LDA, ADD, SUB, and STA do the same thing, BRZ and BRP are the branches, INP and OUT are GET and PRT. Watching the PC advance on its own for the first time teaches more than any trace table.\nWe usually refer to the instruction set as the ISA (Instruction Set Architecture). It defines the interface between software and hardware: which operations the processor can perform and how they are encoded.\nThe conditional branch instructions (JZ, JP, JN) change the PC only if the condition is true. Otherwise, execution continues normally: the PC has already been incremented in the fetch cycle, so the next sequential instruction is executed automatically.\nEncoding instructions # To encode an instruction in hexadecimal, just concatenate the opcode bits with the operand bits:\nLDA 180\\(_{10}\\):\nOpcode of LDA: \\(1_{10} = 0001_2\\) Operand address: \\(180_{10} = 1011\\ 0100_2 = \\text{B4}_{16}\\) Complete instruction (12 bits): \\(\\underbrace{0001}_{\\text{opcode}} \\underbrace{10110100}_{\\text{operand}} = \\mathbf{1\\text{B}4}_{16}\\) ADD 181\\(_{10}\\):\nOpcode of ADD: \\(3_{10} = 0011_2\\) Operand address: \\(181_{10} = 1011\\ 0101_2 = \\text{B5}_{16}\\) Complete instruction: \\(\\underbrace{0011}_{\\text{opcode}} \\underbrace{10110101}_{\\text{operand}} = \\mathbf{3\\text{B}5}_{16}\\) The inverse process (decoding a hexadecimal instruction) is just as straightforward. Just separate the 4 most significant bits (opcode) from the 8 least significant bits (operand):\nExample: decoding \\(\\text{135}_{16}\\) \\(\\text{135}_{16} = 0001\\ 0011\\ 0101_2\\)\nOpcode = \\(0001_2 = 1_{10}\\) → LDA\nOperand = \\(0011\\ 0101_2 = \\text{35}_{16} = 53_{10}\\) → address 53\nDecoded instruction: LDA 53\nThe complete instruction cycle # With the instruction set defined, the flowchart of the instruction cycle is complete. It runs indefinitely until it finds HLT:\n\u0026#x2197; flowchart TD START([\"▶ START\"]) subgraph BUSCA[\"Fetch Cycle\"] B1[\"MAR ← PC\nSends address to the bus\"] B2[\"MDR ← M(MAR)\nReads instruction from MM\"] B3[\"IR ← MDR\nPC ← PC + 1\"] end subgraph EXEC[\"Execute Cycle\"] E1{\"Decode\nopcode in IR\"} E2{\"Need to\nfetch operand?\"} E3[\"MAR ← operand in IR\nMDR ← M(MAR)\"] E4[\"Execute operation\n(ALU or CU)\"] end HLT{\"HLT?\"} STOP([\"⏹ STOP\"]) START --\u003e B1 B1 --\u003e B2 B2 --\u003e B3 B3 --\u003e E1 E1 --\u003e E2 E2 --\u003e|\"Yes\"| E3 E3 --\u003e E4 E2 --\u003e|\"No\"| E4 E4 --\u003e HLT HLT --\u003e|\"No\"| B1 HLT --\u003e|\"Yes\"| STOP Note that the PC is incremented in the fetch cycle, before execution. While the current instruction is being executed, the PC already points to the next one. This matters for the branch instructions: JMP and the conditional variants overwrite the PC during execution, redirecting the program flow.\nStep-by-step execution: LDA # The next two tables follow LDA B4 and then ADD B5 register by register, and that requires keeping five addresses in your head at the same time. The figure below serves as a map for reading. It shows the stretch of memory that the two executions use and the path of each one, in three numbered steps: the PC points to the instruction\u0026rsquo;s cell, the instruction\u0026rsquo;s operand points to the datum\u0026rsquo;s cell, and the datum arrives at the accumulator.\nThe stretch of memory used in the two examples. The instructions are at 02 and 03, the data at B4 and B5; in each panel, the PC points to the instruction (1), the operand points to the datum (2), and the datum arrives at the ACC (3). Let\u0026rsquo;s follow the execution of LDA B4 (the instruction that the previous encoding produced, \\(\\text{1B4}_{16}\\)) with the initial state: PC = 02; MM[02] = \\(\\text{1B4}_{16}\\) (the instruction); MM[B4] = \\(\\text{1A7}_{16}\\) (the datum to load).\nStep Operation in RTL Register state 1 \\(\\text{MAR} \\leftarrow (\\text{PC})\\) MAR = 02 2 \\(\\text{MDR} \\leftarrow M(\\text{MAR})\\) MDR = 1B4 3 \\(\\text{IR} \\leftarrow \\text{MDR}\\) IR = 1B4 4 \\(\\text{PC} \\leftarrow \\text{PC} + 1\\) PC = 03 5 Decoding: 4 MSBs of 1B4 = 0001 → LDA no change 6 Operand B4 → MAR MAR = B4 7 CU activates READ on the control bus no change 8 \\(\\text{MDR} \\leftarrow M(\\text{B4})\\) MDR = 1A7 9 \\(\\text{ACC} \\leftarrow \\text{MDR}\\) ACC = 1A7 ✓ Steps 1 to 4 are the fetch cycle (the same \\(t_0, t_1, t_2\\) of the previous article). Steps 5 to 9 are the execute cycle specific to LDA.\nStep-by-step execution: ADD # Continuing from the state left by LDA: PC = 03; MM[03] = \\(\\text{3B5}_{16}\\); ACC = \\(\\text{1A7}_{16}\\); MM[B5] = \\(\\text{07D}_{16}\\).\nStep Operation in RTL Register state 1 \\(\\text{MAR} \\leftarrow (\\text{PC})\\) MAR = 03 2 \\(\\text{MDR} \\leftarrow M(\\text{MAR})\\) MDR = 3B5 3 \\(\\text{IR} \\leftarrow \\text{MDR}\\) IR = 3B5 4 \\(\\text{PC} \\leftarrow \\text{PC} + 1\\) PC = 04 5 Decoding: 4 MSBs of 3B5 = 0011 → ADD no change 6 Operand B5 → MAR MAR = B5 7 CU activates READ no change 8 \\(\\text{MDR} \\leftarrow M(\\text{B5})\\) MDR = 07D 9 ALU receives ACC = 1A7 and MDR = 07D no change 10 ALU computes: \\(\\text{1A7}_{16} + \\text{07D}_{16}\\) no change 11 \\(\\text{ACC} \\leftarrow \\text{result}\\) ACC = 224 ✓ Check: \\(\\text{1A7}_{16} = 423_{10}\\); \\(\\text{07D}_{16} = 125_{10}\\); \\(423 + 125 = 548_{10} = \\text{224}_{16}\\) ✓\nExecuting branches: JMP and JZ # Branches differ from LDA and ADD in one important respect: instead of fetching a datum for the ALU to process, they change the execution flow through the Program Counter.\nJMP 09 (unconditional branch):\nNormal fetch → the IR receives the JMP instruction; the PC is incremented. Decoding: the CU identifies JMP → signals an unconditional jump. \\(\\text{PC} \\leftarrow 09\\): the operand goes straight to the PC, with no access to MM. The next fetch occurs at address 09. JZ 07 (branch if ACC = 0):\nNormal fetch → IR = JZ 07; the PC is incremented. Decoding: the CU identifies JZ → signals a conditional jump. The ALU checks whether ACC = 0. If ACC = 0: \\(\\text{PC} \\leftarrow 07\\) → next execution at address 07. If ACC ≠ 0: the PC stays incremented → next sequential instruction. The same logic applies to JP (branch if ACC \u0026gt; 0) and JN (branch if ACC \u0026lt; 0): the condition is evaluated by the ALU using the status flags, and the PC is overwritten only if the condition is true.\nAbout flags in this hypothetical processor In real CPUs, conditional branches usually depend on status flags produced by the ALU, such as Zero, Negative, Carry, and Overflow. We have already talked about flags in this article. In this teaching processor, however, the JZ, JP, and JN branches can be understood more simply: they directly test the logical value of the accumulator (ACC = 0, ACC \u0026gt; 0, ACC \u0026lt; 0).\nFrom machine language to assembly # Programming directly in hexadecimal is laborious and error-prone. Assembly language solves this by replacing the binary codes with alphanumeric mnemonics that keep a direct relationship with the machine instructions.\nAspect Details Relationship with the machine 1:1 (one assembly instruction = one machine instruction) Readability Much greater than hexadecimal code Translation Performed by a program called an Assembler Portability Low: specific to each architecture Compiler vs. Assembler Assembler: translates assembly language into machine code for the target architecture, normally preserving a very close correspondence with the processor\u0026rsquo;s actual instructions. Compiler: translates a higher-level language into some form of executable or intermediate representation, which can later be converted into machine code. In teaching models like this one, it is common to treat assembly as having an almost direct relationship with the machine\u0026rsquo;s instructions. In real tools, however, pseudoinstructions, macros, and optimizations can break a strictly 1:1 correspondence.\nStructure of an assembly line # Label Operation Operands ; Comment START: LDA A ; Loads the value of A into the ACC The label is optional and identifies the instruction\u0026rsquo;s address (useful for branches). The operation is the mnemonic. The comment is ignored by the assembler.\nExample: \\(X = A + B - C\\) # In a high-level language, one line. In assembly, four instructions, one for each primitive CPU operation:\nLDA A ; ACC ← (A) ADD B ; ACC ← ACC + (B) SUB C ; ACC ← ACC − (C) STR X ; (X) ← ACC Each line corresponds exactly to one machine instruction. The compiler, when processing the high-level expression, generates a sequence similar to this.\nAssembly in practice # Next, complete programs that illustrate the use of the hypothetical processor\u0026rsquo;s instruction set on real problems. Memory addresses are expressed in hexadecimal.\nPassword check with scrambling # A system stores the password in scrambled form at address 42: the stored value is \\(S/2 - (30)\\), where \\((30)\\) is the content of address 30. The user types their attempt at address 20. The program must check whether the attempt matches and write 1 to 22 (match) or 0 (mismatch).\nAuxiliary addresses: 40 = 0, 41 = 1, 43 = 2.\n; Strategy: scramble what the user typed and compare it with the stored value 00: LDA 20 ; ACC ← user\u0026#39;s attempt 01: DIV 43 ; ACC ← ACC / 2 02: SUB 30 ; ACC ← ACC − (30): now ACC holds the scrambled attempt 03: SUB 42 ; ACC ← computed scrambled value − stored scrambled value 04: JZ 08 ; If ACC = 0 → passwords match; jump to 08 05: LDA 40 ; ACC ← 0 06: STR 22 ; (22) ← 0: wrong password 07: HLT 08: LDA 41 ; ACC ← 1 09: STR 22 ; (22) ← 1: correct password 0A: HLT Notice that all addresses have two hexadecimal digits, and this is not aesthetics: the operand has 8 bits, so the largest possible address is FF. A program that cited an address such as 120 would be outside this machine\u0026rsquo;s memory.\nThe SUB → JZ/JP/JN pattern is the way to implement a comparison on the hypothetical processor: subtract the two values and check whether the result is zero, positive, or negative.\nThe scrambling by division has a flaw This processor\u0026rsquo;s ALU only works with integers, and DIV discards the remainder. This means consecutive passwords collide: if the true password is \\(S = 7\\), the stored value is \\(7/2 - (30) = 3 - (30)\\), and the attempt \\(6\\) produces \\(6/2 - (30) = 3 - (30)\\), exactly the same. The program would accept both.\nThe problem is not in the assembly, it is in the scrambling scheme: any transformation that loses information accepts more than one input. A scrambling that only added, subtracted, or multiplied would not have this effect. It serves as a reminder that integer division is not division: the same care that appears in the sensor average later on.\nThe same thing using GET and PRT The statement says \u0026ldquo;the user types their attempt at address 20\u0026rdquo;. But who puts the value there? In the program above, someone from outside. The processor\u0026rsquo;s repertoire has instructions for exactly this: GET reads from the input port and PRT writes to the output port. With them, the program stops depending on external magic:\n00: GET 20 ; reads the user\u0026#39;s attempt straight into (20) 01: LDA 20 ; ACC ← attempt 02: DIV 43 ; ACC ← ACC / 2 03: SUB 30 ; ACC ← scrambled attempt 04: SUB 42 ; compares with the stored scrambled password 05: JZ 0A ; match → jump to 0A 06: LDA 40 ; ACC ← 0 07: STR 22 ; (22) ← 0 08: PRT 22 ; shows the result (0 = wrong password) 09: HLT 0A: LDA 41 ; ACC ← 1 0B: STR 22 ; (22) ← 1 0C: PRT 22 ; shows the result (1 = correct password) 0D: HLT Two more instructions and the program gains real input and output. The previous version remains valid: in exams and books it is common for the data to appear already \u0026ldquo;placed in memory\u0026rdquo;, and knowing how to read both styles is part of the game.\nPayroll with a salary cap # Compute an employee\u0026rsquo;s total pay:\n$$\\text{Total\\_Pay} = \\min(\\text{Gross\\_Pay} + \\text{Bonus},\\ \\text{Cap}) + \\text{Extra\\_Pay}$$Data: 15 = Gross_Pay, 16 = Bonus, 17 = Cap, 18 = Extra_Pay, 19 = Total_Pay.\n00: LDA 15 ; ACC ← Gross_Pay 01: ADD 16 ; ACC ← Gross + Bonus 02: SUB 17 ; ACC ← (Gross + Bonus) − Cap 03: JP 07 ; If ACC \u0026gt; 0 (exceeded the cap) → jump to 07 04: LDA 15 ; Did not exceed: reload Gross_Pay 05: ADD 16 ; ACC ← Gross + Bonus 06: JMP 08 ; Jump to add the extra pay 07: LDA 17 ; Exceeded: use the Cap as the base 08: ADD 18 ; ACC ← base + Extra_Pay 09: STR 19 ; Total_Pay ← ACC 0A: HLT Average of four sensors # Four sensor readings are at addresses 15, 16, 17, and 18. Compute the average and store it at 13. Address 14 contains the value 2.\n; Division by 4 = two consecutive DIVs by 2 00: LDA 15 ; ACC ← sensor1 01: ADD 16 ; ACC ← ACC + sensor2 02: ADD 17 ; ACC ← ACC + sensor3 03: ADD 18 ; ACC ← ACC + sensor4 (total sum) 04: DIV 14 ; ACC ← ACC / 2 05: DIV 14 ; ACC ← ACC / 2 (total: divided by 4) 06: STR 13 ; average ← ACC 07: HLT In this processor, DIV is an explicit instruction of the repertoire. In a real CPU, dividing by 4 can also be implemented with bit shifts, but that is an implementation detail of the ALU, not a requirement of the assembly program. Dividing by 4 is equivalent to two 1-bit arithmetic right shifts, the same as \u0026gt;\u0026gt; 2 in modern programming languages.\nAverage and count of critical values # Three measurements at addresses 30, 31, and 32. (a) Compute the average and store it at 40. (b) Count how many measurements exceed 50 and store the count at 41.\nAuxiliary addresses: 50 = 0, 51 = 1, 53 = 3, 55 = critical value 50.\n; --- Part A: Average --- 00: LDA 30 ; ACC ← meas1 01: ADD 31 ; ACC ← ACC + meas2 02: ADD 32 ; ACC ← ACC + meas3 03: DIV 53 ; ACC ← ACC / 3 04: STR 40 ; average ← ACC ; --- Part B: Count of values \u0026gt; 50 --- 05: LDA 50 ; ACC ← 0 (initialize counter) 06: STR 41 ; counter ← 0 ; --- Test meas1 \u0026gt; 50 --- 07: LDA 30 08: SUB 55 ; ACC ← meas1 − 50 09: JN 0E ; if meas1 \u0026lt; 50, do not count 0A: JZ 0E ; if meas1 = 50, do not count either 0B: LDA 41 0C: ADD 51 0D: STR 41 ; --- Test meas2 \u0026gt; 50 --- 0E: LDA 31 0F: SUB 55 10: JN 15 11: JZ 15 12: LDA 41 13: ADD 51 14: STR 41 ; --- Test meas3 \u0026gt; 50 --- 15: LDA 32 16: SUB 55 17: JN 1C 18: JZ 1C 19: LDA 41 1A: ADD 51 1B: STR 41 1C: HLT The correct pattern for testing \u0026ldquo;greater than\u0026rdquo; with this processor is:\nsubtract the threshold; discard the negative case with JN; discard the equal case with JZ; only then increment the counter. The same thing using INC The snippet that increments the counter appears three times in the program above:\nLDA 41 ; ACC ← counter ADD 51 ; ACC ← counter + 1 — (51) holds the number 1 STR 41 ; counter ← ACC The repertoire has INC, which adds 1 to the ACC and needs no operand at all:\nLDA 41 ; ACC ← counter INC ; ACC ← counter + 1 STR 41 ; counter ← ACC The gain here is not in instructions: there are three in both versions. It is in data memory: address 51, which existed only to hold the number 1, is no longer needed. On a machine with 256 cells, every saved address counts. The DCR instruction does the symmetric thing, and it is what you use to count down to zero, the pattern of the loop in the next section.\nMultiplication by successive additions # The programs so far have used ADD, SUB, and DIV. The classic case is missing: how do you multiply when there is no multiplication instruction? The answer is a loop that adds the multiplicand as many times as the multiplier says. It is worth building, for two reasons: it is how computers without a multiplication instruction multiplied, like most minicomputers until the end of the 1970s, and it is the first program in this article with a real repetition structure.\nLet\u0026rsquo;s compute \\((62) = (60) \\times (61)\\).\nAuxiliary addresses: 63 = 0, 64 = 1, 65 = working counter.\n; product ← 0; repeat (61) times: product ← product + (60) 00: LDA 63 ; ACC ← 0 01: STR 62 ; product ← 0 02: LDA 61 ; ACC ← multiplier 03: STR 65 ; counter ← multiplier 04: LDA 65 ; ACC ← counter 05: JZ 0D ; counter = 0 → done 06: LDA 62 ; ACC ← partial product 07: ADD 60 ; ACC ← product + multiplicand 08: STR 62 ; product ← ACC 09: LDA 65 ; ACC ← counter 0A: SUB 64 ; ACC ← counter − 1 0B: STR 65 ; counter ← ACC 0C: JMP 04 ; back to the test 0D: HLT Following with \\((60) = 3\\) and \\((61) = 2\\): the counter starts at 2, the product goes through 0, 3, and 6, the counter drops to 1 and then to 0, and the JZ at address 04 ends the execution with \\((62) = 6\\). That is fourteen instructions for one multiplication, and the execution time grows with the value of the multiplier, not with its number of bits. Multiplying by 1000 costs a thousand trips around the loop.\nThe same thing using MPY This processor has MPY in its repertoire, and with it the whole program collapses to four instructions:\n00: LDA 60 ; ACC ← multiplicand 01: MPY 61 ; ACC ← multiplicand × multiplier 02: STR 62 ; product ← ACC 03: HLT Fourteen instructions become four, the three auxiliary addresses disappear, and the execution time stops depending on the values of the operands. The work did not vanish: it went down to the hardware, where a multiplier circuit does in parallel what the loop did in series. It is exactly the trade-off that the CISC vs. RISC discussion presents (more work per instruction versus simpler instructions), seen from the side of whoever writes the program.\nIt is worth knowing how to do it both ways. The loop version is what is asked for when the goal is to understand the operation from the inside; the MPY version is what you write when the instruction exists.\nThe reverse path: STR (Store) # While the LDA and ADD instructions bring data from memory into the CPU, the STR instruction does the exact opposite: it saves the Accumulator\u0026rsquo;s result back into Main Memory. The flow of microevents closes the communication cycle:\n(MAR) \u0026lt;- (IR[address]): The destination address (where the datum will be saved) is copied into the MAR. (MDR) \u0026lt;- (ACC): The already computed datum leaves the Accumulator and goes to the MDR (the output \u0026ldquo;waiting room\u0026rdquo;). (MM(MAR)) \u0026lt;- (MDR): The Control Unit issues the write signal over the control bus, and the memory physically records the datum at the specified address. This flow closes the complete communication cycle between CPU and memory: fetch, process, and store the result.\nInside the ALU: how does silicon calculate? # Up to this point, we have treated the ADD operation without much detail. The decoder sends a signal and the ALU simply \u0026ldquo;adds\u0026rdquo; the values that arrived from memory. But how can a piece of silicon do math?\nThe answer is that the ALU does not understand numbers; it understands electricity. It is built from thousands of microscopic switches organized into circuits called logic gates (such as AND, OR, NOT, and XOR). Each gate receives electrical signals (zeros and ones) and emits an answer based on a simple logical rule. If you have a bit of difficulty, I recommend this series' article on Boolean Algebra.\nTo actually perform the addition when the ADD instruction is executed, the CPU combines these logic gates into mathematical blocks:\nHalf Adder: A simple circuit that uses logic gates to add two individual bits, producing a result and a \u0026ldquo;carry\u0026rdquo;. Its flaw is that it cannot receive a carry from a previous operation. Full Adder: The evolution of the circuit, capable of adding three bits (the two operands plus the carry from the neighboring column). By lining up several Full Adders side by side, the ALU can add entire words of 12, 32, or 64 bits, propagating the carry from the least significant bit to the end. This cascade arrangement (ripple-carry) is the simplest, but the delay grows with the number of bits, which is why real processors use faster variants, such as the carry-lookahead adder. This is how the abstract Assembly command becomes a real electrical flow.\nBit shifts # Besides using branches to optimize the program flow, the CPU also has \u0026ldquo;shortcuts\u0026rdquo; implemented directly in hardware to speed up heavy math. Instead of triggering the adders multiple times to do simple multiplications or divisions, the ALU includes shift operations. They are equivalent to multiplying or dividing by powers of 2, but executed almost instantly by \u0026ldquo;pushing\u0026rdquo; the bits sideways:\n$$\\text{Shift left}\\ n\\ \\text{bits} = \\times\\ 2^n \\qquad \\text{Shift right}\\ n\\ \\text{bits} = \\div\\ 2^n$$We have already covered this in an article on bitwise operators, which explored bit-by-bit operations. Let\u0026rsquo;s look at a practical example to fix the idea.\nFor \\(5_{10} = 0101_2\\):\nOperation Binary result Decimal result Shift left 1 bit \\(1010_2\\) \\(10_{10}\\) (\\(5 \\times 2\\)) ✓ Shift right 1 bit \\(0010_2\\) \\(2_{10}\\) (\\(5 \\div 2\\), truncated) ✓ The sensor average example (dividing the sum by 4 with two DIV 14 where (14) = 2) is exactly two consecutive right shifts.\nScope of this remark The example above describes a common mechanism in real ALUs. It does not automatically mean that this article\u0026rsquo;s hypothetical processor has explicit SHL or SHR instructions in its assembly repertoire.\nThe execution trace (tracer) # To visualize the CPU\u0026rsquo;s behavior over time, it is useful to build a trace table that records the state of the main registers at each fetch and execute phase.\nThe table below is a didactic construction, inspired by the operation of the hypothetical processor, to follow the sequence LDA 20, ADD 21, STR 22 assuming MM[20] = 5 and MM[21] = 3:\nStep Phase PC MAR MDR IR ACC 1 Fetch 01 00 LDA 20 LDA 20 0 2 Execute 01 20 5 LDA 20 5 3 Fetch 02 01 ADD 21 ADD 21 5 4 Execute 02 21 3 ADD 21 8 5 Fetch 03 02 STR 22 STR 22 8 6 Execute 03 22 8 STR 22 8 This kind of table works like an \u0026ldquo;X-ray\u0026rdquo; of the instruction cycle.\nObserve the perfect flow: in step 6, the value 8, which was in the Accumulator, was thrown to the MDR to be physically written at address 22 of memory. The CPU works to the rhythm of the clock.\nConclusion and next articles # The hypothetical processor condenses the essential mechanisms of any real processor: an instruction repertoire encoded in bits, a cycle that fetches and executes each instruction in sequence, and branches that alter that flow based on conditions. Assembly reveals the direct correspondence between those instructions and the code the hardware executes.\nIn the next article, we will move on to the parallel execution of multiple instructions with the pipeline technique. We will see how it multiplies performance, which hazards can interrupt that flow, and how the control unit coordinates everything to keep execution flowing.\nSee you then!\n","date":"2026-04-21","externalUrl":null,"permalink":"/2026/04/cpu-hypothetical-processor/","section":"Blog","summary":"A hypothetical 12-bit processor with 15 instructions serves as a model for following the complete instruction cycle. We execute LDA and ADD register by register, encode instructions in hexadecimal, and write assembly programs with branches and loops.","title":"CPU in Action: The Hypothetical Processor and the Fundamentals of Assembly","type":"posts"},{"content":"","date":"2026-04-21","externalUrl":null,"permalink":"/tags/instruction-cycle/","section":"tags","summary":"","title":"instruction cycle","type":"tags"},{"content":" In the previous two articles, we built the foundation of the memory system: the memory hierarchy that organizes DRAM, SRAM, and secondary storage, and the cache that acts as a buffer between the processor and main memory. Now we reach the heart of the system: the Central Processing Unit (CPU).\nThe CPU is the component that actually computes: it performs arithmetic operations, makes logical decisions, fetches instructions from memory, and coordinates all the other subsystems. Modern processors are manufactured on a single chip, with billions of transistors, and operate at gigahertz frequencies. But the essence of the CPU remains the same as in the first computers: fetch an instruction, decode it, and execute it, over and over, until the system is turned off.\nIn this article you will learn:\nwhat the two basic functions of the CPU are and what it accomplishes in terms of primitive operations; what a machine instruction is and how it is structured in bits; which components make up the CPU and what role each one plays; what status flags are and why they exist; how the clock synchronizes everything and why frequency is not a synonym for performance; the microevents of the fetch cycle in RTL notation; the differences between CISC and RISC architectures. The CPU in the real world # The CPU is the component that actually executes instructions, processes data, and coordinates the operation of the system. In current computers, it usually comes in a single package connected to the motherboard, although internally it may contain one or more silicon dies, besides caches and specialized functional units.\nDespite the enormous physical evolution (billions of transistors, gigahertz frequencies, and multiple cores), the fundamental idea remains the same: fetch an instruction, decode it, and execute it.\nThe processor in the real world When we look at the specification of a real processor, we see not only its logical organization but also physical and integration characteristics: manufacturing technology, number of transistors, type of package, socket, number of cores, cache sizes, and the presence of units specialized in different kinds of computation, such as integers, floating point, and vectors.\nWhat the CPU does # The CPU has two fundamental functions:\nData processing: performing arithmetic operations (add, subtract, multiply, divide), logical operations (AND, OR, XOR), data movement, branches, and input/output operations. Control: coordinating the operation of all the components of the system (main memory, I/O devices, and the other subsystems). In practice, every complex computation is decomposed into primitive operations: add, subtract, move data between storage locations, transfer data to output devices. A compiler, when processing an expression such as X = A + B*C, generates a sequence of primitive instructions that the CPU executes one by one:\nT = B * C ; 1st instruction X = A + T ; 2nd instruction The CPU never sees the high-level expression: only the elementary instructions that the compiler produced from it.\nThe figure below shows a simplified diagram of the CPU, highlighting its main components and how they connect to main memory. Note that the CPU is a complex system, but we can understand its basic organization from its functional blocks and the data and control flows between them.\nSimplified diagram of the CPU We will detail these components throughout the article, but first we need to understand what a machine instruction is.\nMachine instructions # A machine instruction is a sequence of bits that encodes exactly what the CPU must do. It has two main fields:\nOperation code (Opcode): specifies which operation to execute Operands: the data to operate on or the addresses from which to obtain them For example, a 12-bit instruction with 4 bits for the opcode and 8 bits for the operands could be structured like this:\n| Opcode (4 bits) | Operand 1 (4 bits) | Operand 2 (4 bits) | 0 1 0 1 0 0 0 1 0 0 1 0 The operation above could, for example, represent ADD R1, R2: add the values of registers R1 and R2 and store the result in an implicit destination register (such as the ACC).\nWith \\(n\\) opcode bits, we have \\(2^n\\) possible operations. With 4 bits, for example, we get \\(2^4 = 16\\) distinct operations.\nThe program lives in main memory An executable program is a sequence of machine instructions stored in consecutive cells of the MM. To start execution:\nthe instructions must be in successive cells of the MM; the address of the first instruction must be in the PC (Program Counter); the CPU executes the instructions in address order, except for explicit branches. Word size and registers # In classic architectures, the processor\u0026rsquo;s word size usually coincides with the width of the general-purpose registers and with the number of bits handled at once by the ALU. This does not mean, however, that all the CPU\u0026rsquo;s registers necessarily have the same width.\nSpecialized registers can have different sizes, according to their function. For example:\nthe PC and the MAR follow the size of the address; the MDR / MBR follows the width of the data transfer with memory; the general-purpose registers and the ACC normally follow the processor\u0026rsquo;s word. The impact of word size # Word size directly influences the amount of data the CPU can transfer and process at a time. In a simple example, if two 16-bit values need to be added:\nin a system with an 8-bit word, each value will have to be handled in two parts; in a system with a 16-bit word, each value can be handled at once. This does not mean that a processor with a larger word will always be exactly twice as fast, but it helps us understand why architectures with larger words tend to reduce the number of accesses and steps needed for certain operations.\nTo add \\(A = \\text{3A25}_{16}\\) and \\(B = \\text{172C}_{16}\\) (16-bit values):\nSystem Word width MM accesses Relative time System 1 8 bits 2 accesses (8 bits at a time) \\(T_1\\) System 2 16 bits 1 access (16 bits at once) \\(T_2 \\approx T_1/2\\) This explains the historical evolution: from 8- and 16-bit processors to the 32- and 64-bit ones found today in any personal computer or smartphone.\nCPU components # Internally, the CPU is composed of two major functional blocks: the Control Unit (CU) and the Arithmetic Logic Unit (ALU), connected by registers and internal buses.\n\u0026#x2197; graph TB subgraph UCP[\"Central Processing Unit\"] subgraph CTR[\"Control Unit\"] REL[\"Clock\"] UC[\"CU — Control Unit\"] DEC[\"Decoder (Opcode → operation)\"] CI[\"PC — Next instruction\"] RI[\"IR — Current instruction\"] REM[\"MAR — Address to MM\"] RDM[\"MDR / MBR — Data from MM\"] end subgraph PROC[\"Arithmetic Logic Unit\"] UAL[\"ALU\"] ACC[\"ACC — Accumulator\"] REGS[\"General-purpose registers\"] FLAGS[\"Flags Register (N, Z, V, C)\"] end REL --\u003e UC UC --\u003e DEC RI --\u003e DEC DEC --\u003e UAL CI --\u003e REM RDM --\u003e RI UAL \u003c--\u003e ACC UAL \u003c--\u003e REGS UAL --\u003e FLAGS end MP[(\"Main Memory\")] REM --\u003e|\"Address Bus\"| MP RDM \u003c--\u003e|\"Data Bus\"| MP UC -.-\u003e|\"Control Bus\"| MP Component Alternative name Function ALU Arithmetic Logic Unit Performs arithmetic and logic operations Registers Registers Temporarily store data, addresses, and instructions ACC Accumulator Special register; implicit result in older architectures IR Instruction Register Stores the instruction being executed PC Program Counter Stores the address of the next instruction MAR Memory Address Register Stores the address for accessing the MM MDR MBR (Memory Buffer Register) Data buffer between the CPU and the MM CU Control Unit Generates the control signals for all components Decoder Decoder Identifies the operation from the opcode bits Clock Clock Generates pulses that synchronize all operations Register size and processor word General-purpose registers usually have the width of the processor\u0026rsquo;s word: this coincidence is where the habit of calling a machine a \u0026ldquo;32-bit processor\u0026rdquo; comes from. As seen above, this does not hold for all of them: specialized registers follow the quantity they handle, not the word.\nThe x86 shows well how this width is organized in layers. In the Intel 486, with a 32-bit word, the general-purpose register is EAX. Its 16 least significant bits can be accessed with the name AX, inherited from the earlier 16-bit processors; and AX, in turn, splits into a high and a low part (AH and AL, 8 bits each), which already existed in the 8086 and were designed to ease the migration of code from 8-bit processors. That is three generations of compatibility stacked inside the same register, and that is why AL, AX, and EAX appear side by side in modern assembly code.\nStatus flags # The ALU calculates results and also records properties of the last result in special bits called flags, grouped into what is usually called the status register. These bits are consulted by conditional branch instructions (if, while, etc.) to decide whether the program flow should change.\nA well-known historical example is the Motorola 68000 flag set:\nFlag Meaning Condition for being set N Negative Result \u0026lt; 0 Z Zero Result = 0 V Overflow Register capacity exceeded C Carry Carry generated in an arithmetic operation X Extend Carry extension This set should not be read as \u0026ldquo;the universal current standard\u0026rdquo;, but as a good historical example for understanding the role of flags in a CPU.\nFor example, after computing A - B, the CPU checks the Z flag to know whether the result was zero, which is equivalent to A == B. A conditional branch instruction uses these flags to decide which instruction to execute next.\nThe N flag alone does not answer A \u0026lt; B The temptation is natural: if A - B came out negative, then \\(A \u003c B\\). In two\u0026rsquo;s complement, however, subtraction can overflow, and then the sign of the result lies.\nTake \\(A = -100\\) and \\(B = 100\\) in 8 bits. The true result, \\(-200\\), does not fit in the range \\([-128, +127]\\); what ends up stored is \\(56\\), positive. The N flag is 0, even though \\(A\\) is clearly less than \\(B\\). What the CPU does in this case is set the V flag, exactly the one that appears two lines above in the table and exists to signal this situation.\nThe correct condition for a signed comparison is \\(A \u003c B \\iff N \\oplus V\\) (N exclusive-or V): the sign of the result, corrected when there was overflow. This is why processors offer two sets of conditional branches: one for signed numbers, which consults N and V, and another for unsigned numbers, which consults the C flag. Choosing the wrong set is a classic source of bugs.\nHowever, before it can calculate anything or check flags, the CPU must answer the most basic question when reading memory: \u0026ldquo;What does this sequence of zeros and ones mean?\u0026rdquo;. This is where the decoder comes into play.\nThe decoder # The decoder is a combinational circuit of the \\(n\\)-to-\\(2^n\\) type: each combination of the opcode bits activates exactly one output, which corresponds to the operation to perform.\n$$n\\ \\text{opcode bits} \\implies 2^n\\ \\text{possible distinct operations}$$With 4 bits, for example, 16 outputs; with 8 bits, 256 outputs. Only the output corresponding to the received code is activated; the others remain inactive. This circuit is what turns the opcode bits into electrical signals that configure the ALU for the correct operation.\nThe ALU: Arithmetic Logic Unit # Once the decoder has \u0026ldquo;translated\u0026rdquo; the instruction and prepared the electrical paths of the system, the stage is set for the processor\u0026rsquo;s brute force to act. The ALU obtains the data from the registers and deposits the results back into them. It is the ALU that does all the computational work proper:\nCategory Operations Integer arithmetic Addition, subtraction, multiplication, division Floating point Same operations on real numbers (co-processor, now integrated) Bitwise logic AND, OR, XOR, NOT Bit shifting Shift left (\\(\\times 2\\)), Shift right (\\(\\div 2\\)) Increment/Decrement \\(+1\\), \\(-1\\) The ALU does not access memory directly: it only operates on values that are already in the registers. It is up to the instruction cycle (next section) to ensure that the right data are in the right registers before the ALU operates.\nThe clock and the cycles # The clock is a pulse generator that synchronizes all CPU operations. The frequency indicates how many cycles occur per second:\n$$f = \\frac{1}{T} \\qquad \\Longleftrightarrow \\qquad T = \\frac{1}{f}$$ Processor (example) Frequency Cycle period M1 25 MHz \\(T = 1/(25 \\times 10^6) = 40\\ \\text{ns}\\) M2 400 MHz \\(T = 1/(400 \\times 10^6) = 2.5\\ \\text{ns}\\) Frequency ≠ performance A processor with a higher frequency does not necessarily execute more instructions per second. Pipeline, cache, architecture, and the average number of cycles per instruction (the CPI, for Cycles Per Instruction) also determine real performance.\nThe relationship between these factors fits in one line, in the so-called performance equation:\n$$T_{CPU} = \\frac{I \\times CPI}{f}$$where \\(I\\) is the number of instructions executed, \\(CPI\\) the average number of cycles each one consumes, and \\(f\\) the clock frequency. The three factors weigh equally in the result: doubling the frequency and doubling the CPI comes out exactly the same.\nWith the equation in hand, the claim above stops being an allegation and becomes arithmetic. A 3 GHz processor with a CPI of 2 spends \\(2/(3 \\times 10^9) \\approx 0.67\\) ns per instruction; a 2 GHz one with a CPI of 1 spends \\(1/(2 \\times 10^9) = 0.50\\) ns. The second executes 33% more instructions per second at two thirds of the frequency.\nThe megahertz myth This calculation explains one of the best-known episodes in the industry. In the early 2000s, Intel bet on frequency as a selling point, and the Pentium 4 reached 3.8 GHz. The price was a very long pipeline (from 20 to 31 stages, depending on the version) that inflated the CPI and wasted many cycles on every mispredicted branch. The AMD Athlon 64, at 2.4 GHz, beat it in a good share of applications. Apple, then in its PowerPC era, ran an advertising campaign on this, and the phenomenon became known as the megahertz myth. Intel ended the line and backed off with the Core architecture, with a lower frequency and a much better CPI, and the number of gigahertz never again became the main argument of a campaign.\nEach clock cycle can be subdivided into subcycles (\\(t_0, t_1, t_2, \\ldots, t_n\\)), which correspond to internal microevents of the instruction cycle. The figure below shows this subdivision within the block diagram of the control unit: the clock pulses enter on one side, and each subcycle triggers one of the microevents that make up the execution of an instruction.\nBlock diagram of the CU, showing the clock subcycles To get around the frequency limit and increase performance, processors have evolved their execution modes. Historically, CPUs operated in a strictly sequential mode (an instruction only started after the previous one finished). Today, we operate with concurrent execution (pipeline), where several instructions overlap in different stages (like an assembly line); parallel execution, with multiple cores processing instructions simultaneously; and vector execution, where a single instruction operates on multiple data at the same time.\nThe instruction cycle # The CPU operates in a perpetual loop:\n$$\\text{Instruction cycle} = \\text{Fetch cycle} + \\text{Execute cycle}$$In practice, the choreography of this perpetual loop divides the CPU into two distinct roles. In the first stage, the processor acts as a reader looking for its next task; in the second, it becomes the executor:\nPhase What happens Fetch cycle Fetches the instruction from the MM → stores it in the IR → decodes it Execute cycle Generates control signals → executes in the ALU or accesses devices The figure below shows a simplified flowchart of the instruction cycle, with the fetch, interpretation, and execution of an instruction.\nInstruction cycle Hardwired and microprogrammed control # To generate the control signals of the instruction cycle, there are two classic approaches to designing the Control Unit:\nhardwired control: the signals are produced by fixed logic circuits; it tends to be faster, but also less flexible; microprogrammed control: the microoperations are described in an internal control memory; it tends to simplify the design of the CU, although it is normally slower than fully hardwired control. This distinction matters because it shows that the CPU does not \u0026ldquo;understand\u0026rdquo; instructions by magic: it depends on an internal mechanism that translates each instruction into an organized sequence of control signals.\nMicroevents of the fetch cycle # The fetch cycle unfolds into three consecutive micro-timings, described in RTL notation (the same one we used for memory operations in the article on the memory hierarchy):\nSubcycle Microevent in RTL Description \\(t_0\\) \\(\\text{MAR} \\leftarrow (PC)\\) The address of the next instruction goes to the MAR; it is sent over the address bus to the MM \\(t_1\\) \\(PC \\leftarrow PC + N\\) and \\(\\text{MDR} \\leftarrow M(\\text{MAR})\\) The PC is incremented (it points to the following instruction) while the MM reads the instruction and places it in the MDR \\(t_2\\) \\(\\text{IR} \\leftarrow \\text{MDR}\\) The instruction is transferred from the MDR to the IR, where it will be decoded by the CU Here, \\(N\\) represents the number of cells occupied by the instruction. In fixed-length instruction architectures, this increment is usually constant; in more complex architectures, it depends on the actual size of the fetched instruction.\nNote that \\(t_1\\) has two simultaneous operations: the PC increment and the MM read can occur in parallel because they use distinct resources, the PC and the ALU (for the increment) versus the bus and the MM (for the read).\nInstruction formats # An instruction can contain a variable number of operands, depending on the architecture. The three main formats differ in how many explicit values the instruction references:\nFormat Semantics Assembly example 3 operands \\(\\text{Op3} \\leftarrow \\text{Op1} \\oplus \\text{Op2}\\) ADD R1, R2, R3 → R3 = R1 + R2 2 operands \\(\\text{Op1} \\leftarrow \\text{Op1} \\oplus \\text{Op2}\\) ADD R1, R2 → R1 = R1 + R2 1 operand \\(\\text{ACC} \\leftarrow \\text{ACC} \\oplus \\text{Op}\\) ADD B → ACC = ACC + (B) The 1-operand format is characteristic of accumulator architectures: the ACC is always one of the implicit operands, which simplifies the decoder circuit at the cost of less expressiveness per instruction.\nThe opcode size can be fixed (simpler and faster implementation, but it limits the number of operations) or variable (more flexible, more complex decoding). CISC architectures tend to use variable formats; RISC, fixed ones.\nCISC vs. RISC # This discussion about instruction design, whether formats should be variable and full of options or fixed and lean, created two schools of thought that clashed for decades:\nCharacteristic CISC RISC Full name Complex Instruction Set Computers Reduced Instruction Set Computers Philosophy Many instructions, some of them complex Few instructions, simpler and more uniform Instruction format Varied (different sizes) Uniform (fixed size) Addressing modes More numerous More restricted Historical focus Expressiveness per instruction Ease of implementation, pipelining, and fast execution The intuition behind CISC was to reduce the number of instructions needed to represent a program. The intuition behind RISC was to simplify the instructions to make execution faster and more predictable.\nIn practice, modern processors mix ideas from both schools: for example, the x86 family keeps a historically CISC interface, but internally translates the instructions into simpler micro-operations. This table is just the map of the territory. We will return to CISC and RISC in depth in the last article of the series, where the comparison gains data and the conclusion is less obvious than it seems.\nThe article after this one explores in detail how a hypothetical processor executes concrete instructions, step by step.\nPractical examples # With all the components in mind, we can size the registers and buses of a concrete machine from its specifications:\nExample: complete register design A machine must address 4 M cells of main memory, with each cell storing 2 bytes. Each instruction contains: an operation code (at most 64 operations), one MM address operand, and one register operand (the machine has 16 registers).\nDetermine: (a) the size of the MAR and the address bus; (b) the instruction size in bits and in cells; (c) the size of the MDR and the DB to fetch the instruction in a single access; (d) the accesses needed if 1 cell at a time; (e) the sizes of the IR and the PC.\n(a) MAR / AB: \\(N = 4\\text{M} = 2^{22}\\) cells → \\(E = 22\\) bits → MAR = AB = \\(\\mathbf{22}\\) bits\n(b) Instruction size:\nOpcode: 64 operations \\(= 2^6\\) → \\(\\mathbf{6}\\) bits Operand (address): 22 bits Operand (register): 16 registers \\(= 2^4\\) → \\(\\mathbf{4}\\) bits Total: \\(6 + 22 + 4 = \\mathbf{32\\ \\text{bits}}\\) = 2 cells (each cell = 16 bits) (c) MDR / DB for 1 access: To obtain the whole instruction in 1 access → MDR = DB = \\(\\mathbf{32}\\) bits\n(d) Accesses if 1 cell at a time: Instruction = 32 bits; cell = 16 bits → \\(\\mathbf{2}\\) accesses\n(e) IR and PC:\nIR = \\(\\mathbf{32}\\) bits (stores the complete instruction) PC = \\(\\mathbf{22}\\) bits (addresses the whole MM) The next two examples combine concepts seen here with concepts of memory hierarchy and cache seen in the previous articles, showing how the design of the CPU and of the memory are interlinked.\nExample: from maximum capacity to the data bus A machine can address 16 M cells. Each cell stores one word, and each instruction is the size of one word. The operation code supports at most 256 distinct operations and the operand is an MM address. Four fetch cycles are needed to obtain a complete instruction.\nDetermine: size of the PC, total capacity, MAR, IR, AB, and DB.\n\\(N = 16\\text{ M} = 2^{24}\\) cells → PC = MAR = AB = \\(\\mathbf{24}\\) bits\nOpcode: \\(256 = 2^8\\) → 8 bits. Operand: 24 bits. Instruction = \\(8 + 24 = 32\\) bits = size of one cell \\((M)\\).\nIR = \\(\\mathbf{32}\\) bits (stores the whole instruction for decoding).\n\\(T = 2^{24} \\times 32 = \\mathbf{512\\ \\text{Mbits}}\\ (= 64\\ \\text{Mbytes})\\)\nDB: instruction = 32 bits in 4 fetch cycles → DB = \\(32/4 = \\mathbf{8}\\) bits\nExample: two-operand instruction and double fetch cycle A machine has 256 M cells, with 512 distinct instructions. Each instruction contains an operation code and two MM address operands. The data bus has a width equal to half the size of an instruction.\nDetermine: MAR, IR, MDR, total capacity, and describe the fetch cycle.\n\\(N = 256\\text{ M} = 2^{28}\\) → MAR = \\(\\mathbf{28}\\) bits\nOpcode: \\(\\log_2 512 = 9\\) bits. Each operand: 28 bits. Instruction = \\(9 + 28 + 28 = \\mathbf{65\\ \\text{bits}}\\) → IR = MDR = \\(\\mathbf{65}\\) bits\n\\(T = 2^{28} \\times 65 = 256 \\times 65\\ \\text{Mbits} = \\mathbf{16{,}640\\ \\text{Mbits}}\\)\nDB = \\(65/2 \\approx 32\\) bits → 1 bit is left over → \\(\\mathbf{3}\\) fetch cycles for the complete instruction\nConclusion and next articles # We now understand what the CPU is and how it is organized. We know each component (ALU, CU, registers, decoder, clock) and how the fetch cycle mobilizes them in sequence to bring an instruction from memory to the IR. But describing components and the fetch cycle is only half of the story. We have not yet seen what happens during the execute cycle: how the CPU actually processes a specific instruction, operating on concrete data.\nIn the next article, we will explore a hypothetical 12-bit processor with a limited instruction set. It will serve as a model for understanding the complete instruction cycle, the step-by-step execution of instructions such as LDA (load) and ADD (add), and the fundamentals of assembly, the low-level language that communicates directly with the hardware.\nSee you then!\n","date":"2026-04-16","externalUrl":null,"permalink":"/2026/04/cpu-components-instruction-cycle/","section":"Blog","summary":"A 2 GHz processor can execute more instructions per second than a 3 GHz one. To understand why, we open up the CPU: ALU, control unit, registers, flags, decoder, and clock, along with the microevents of the fetch cycle in formal notation.","title":"CPU: Components, Instruction Cycle, and What Happens Inside the Processor","type":"posts"},{"content":"","date":"2026-04-16","externalUrl":null,"permalink":"/tags/processador/","section":"tags","summary":"","title":"processador","type":"tags"},{"content":"","date":"2026-04-16","externalUrl":null,"permalink":"/tags/processor/","section":"tags","summary":"","title":"processor","type":"tags"},{"content":"","date":"2026-04-14","externalUrl":null,"permalink":"/tags/cache/","section":"tags","summary":"","title":"cache","type":"tags"},{"content":" In the previous article, Memory Hierarchy, we saw that DRAM is too slow for the processor and that SRAM, although fast, is too expensive for large volumes. We ended with the principles of temporal and spatial locality: the empirical observation that real programs tend to reuse the same data and to access neighboring addresses. These are exactly the principles that make the cache memory viable, and that is what we are going to explore now.\nIn this article you will understand:\nthe speed bottleneck between CPU and DRAM and why it grows over time; how the principles of locality translate into hit and miss; the three types of mapping that define where a block of MM can go in the cache; the algorithms that decide which block to replace when the cache is full; the write policies and how they keep cache and MM consistent; the physical organization of memory in chips: 1D versus 2D. The CPU/MM Speed Problem # DRAM transfers data to the CPU at a speed lower than the one at which the processor can consume it. When this happens, the CPU has to wait, inserting wait states (waiting cycles) and sitting idle for part of the time.\nHistorically, processor performance has evolved faster than the latency and bandwidth of main memory. This mismatch was one of the main reasons for the appearance of cache memory.\nIt even has a name: the memory wall, a phrase made famous by the paper Hitting the Memory Wall: Implications of the Obvious, by Wulf and McKee, in 1995. And there are numbers behind it. From the mid-1980s on, processor performance started to grow at around 50% per year, while DRAM latency improved by something like 7% per year. Two different exponential rates do not drift apart slowly. That is why, in a current processor running at a few gigahertz, a main memory access costs on the order of two hundred cycles: two hundred opportunities to do work that the CPU spends waiting. Memory got better, but the processor got better much faster, and the gap has never stopped growing.\nThe figure below shows this divergence since 1980. The vertical axis is logarithmic: each grid line is worth ten times the previous one, and that is the only reason the two curves fit on the same chart. On a common scale, the memory curve would be a line practically glued to zero.\nPerformance of the processor and of main memory since 1980, normalized to 1 in the initial year. The curves are a model built from the average annual growth rates published by Hennessy and Patterson, not year-by-year measurements. Two things can be read from the figure besides the gap itself. The first is that it was already tens of times when the term was coined, in 1995. The second is the processor curve flattening from the mid-2000s: the performance gain per core slowed down, for reasons that the last article of the series explains and that pushed the industry toward multicore processors.\nWho invented the cache The idea appears in 1965, in a two-page paper by Maurice Wilkes called Slave Memories and Dynamic Storage Allocation. Wilkes proposed a small, fast memory, which he called a slave memory, keeping copies of what main memory held, managed entirely by the hardware, without the program knowing it existed. That is the definition of a cache, three years before the name caught on.\nThe first commercial machine to ship with one was the IBM System/360 Model 85, in 1968, with 16 KB of high-speed buffer, the term IBM used. It was around then that \u0026ldquo;cache\u0026rdquo; became the standard word, borrowed from the French cacher, to hide: a memory whose purpose is to be invisible to the programmer.\nIn theory, a simple solution would be to build the entire main memory with SRAM, which is much faster. In practice, this is unfeasible: SRAM takes up more area on the chip and costs much more per stored bit. The engineering solution was to place a small, very fast memory between the CPU and the DRAM: the CPU cache.\nWhy does the cache work? The cache is only efficient because real programs exhibit locality of reference. Without this empirically observed behavior, the cache would bring no performance gain.\nPrinciples of Locality # The efficiency of the cache depends on an observable behavior of real programs called locality of reference. Two complementary principles describe this behavior:\nPrinciple Definition Practical example Temporal If the program accesses word \\(X\\), there is a high probability of accessing \\(X\\) again soon Control variable in a for loop Spatial If the program accesses word \\(X\\), there is a high probability of accessing nearby words next Sequential instructions; consecutive elements of an array The cache exploits both: on a miss, it loads a whole block of contiguous cells (spatial locality) and keeps that block in its slots for as long as possible (temporal locality). For most programs, this means that the vast majority of subsequent accesses result in a hit, with no need to search the DRAM. But how exactly does the CPU decide whether the data is in the cache or whether it has to \u0026ldquo;go down\u0026rdquo; to main memory?\nHow the Cache Works # The rule of computer architecture is: the CPU always tries to fetch instructions and data from the cache first. On each access, the hardware checks whether the desired block is already present in it:\n\u0026#x2197; graph TD A[\"CPU requests instruction or data\"] B{\"Block present\nin the cache?\"} H[\"HIT\nData transferred cache→CPU\nat high speed\"] M[\"MISS\nWhole block transferred MM→cache\nthen word→CPU\"] F[\"Data delivered to the CPU\"] A --\u003e B B --\u003e|\"Yes\"| H --\u003e F B --\u003e|\"No\"| M --\u003e F Situation What happens Transfer granularity Hit Data found in the cache Word cache → CPU Miss Data not found Block MM → cache; then word → CPU Block transfer on a miss exploits spatial locality: if a word was needed now, there is a good chance that neighboring words will be accessed next too.\nWhat a Hit Rate Is Worth # The fraction of accesses that result in a hit is called the hit rate, and is usually represented by \\(h\\). It is the number that decides whether the cache was worth it, because the average access time seen by the processor is a weighted average of the two possible paths:\n$$T_m = h \\cdot T_c + (1 - h) \\cdot T_{MM}$$where \\(T_c\\) is the cache access time and \\(T_{MM}\\) the main memory access time. It is worth putting numbers on this, using the orders of magnitude from the previous article, \\(T_c = 1\\) ns and \\(T_{MM} = 80\\) ns:\n$$T_m = 0.95 \\times 1 + 0.05 \\times 80 = 0.95 + 4 = 4.95\\ \\text{ns}$$With a 95% hit rate, the processor sees a 4.95 ns memory, about 16 times faster than plain DRAM, using a cache that holds a tiny fraction of the total. This result is what justifies the existence of the cache.\nThe same calculation shows why cache design is so sensitive: at a 90% hit rate the average time rises to 8.9 ns, almost double. Every percentage point of hit rate lost is costly, because the term it feeds is the slower of the two.\n80 ns\nPlain DRAM\nmain memory access time 4.95 ns\n95% hit rate\naverage time seen by the processor, about 16 times lower 8.9 ns\n90% hit rate\nfive fewer points of hit rate almost double the time Cache Types and Levels # A useful distinction is this:\nwhen we talk about cache memory in this article, the main focus is the processor cache, built to reduce the cost of accessing main memory; when we talk about speeding up disk access, in current systems the topic is commonly found under names such as file system cache or page cache in RAM, and not necessarily as \u0026ldquo;disk cache\u0026rdquo; in the classic sense. But the general idea is always the same: use a faster layer to cushion the cost of accessing a slower layer.\nType Accelerated relationship Typical implementation CPU cache CPU ↔ Main Memory SRAM File system cache RAM ↔ secondary storage DRAM Level Typical location Speed Typical capacity L1 Inside the processor core Maximum Tens of KB L2 Inside the processor Very high Hundreds of KB to a few MB L3 Inside the processor, usually shared High Several MB Split cache: in many architectures, instructions and data are kept in separate caches, which reduces conflicts and improves internal parallelism.\nHistorical evolution of the cache levels In older architectures, it was common for only the L1 cache to sit inside the processor, while the L2 could be outside it, for example on the motherboard or in a separate chip. In current processors, it is most common for L1, L2, and L3 to be integrated into the processor itself. In many cases, L1 and L2 are private to each core, while the L3 is shared. The figures below illustrate this evolution:\nEvolution of the cache levels Main components of a multicore computer The Four Cache Design Elements # Designing a cache memory requires hardware engineers to balance cost, complexity, and performance. This balance involves four elements:\n# Element Issue involved 1 Size What is the total capacity? (affects cost, hit rate, and access time) 2 Mapping function How is a block of MM associated with a cache line? 3 Replacement algorithm Which block to remove when a miss occurs and there is no free line? 4 Write policy How to keep cache and MM consistent on write operations? Essential terminology:\nBlock: unit of transfer between MM and cache; contains \\(K\\) contiguous cells Line: space in the cache that holds a block \\(B\\) = total number of blocks in MM \\(Q\\) = number of lines in the cache, where \\(Q \\ll B\\) Mapping Functions # Direct Mapping # Each block of MM has a single possible cache line, determined by:\n$$\\text{cache line} = \\text{block number} \\bmod Q$$The figure below illustrates direct mapping, where MM blocks are mapped to specific cache lines. Note that different blocks can compete for the same line, which can lead to frequent conflicts.\nExample of direct mapping Division of the \\(E\\)-bit address:\nField No. of bits Function Tag \\(E - \\log_2 Q - \\log_2 K\\) Identifies which of the \\(B/Q\\) blocks is in that line Line no. \\(\\log_2 Q\\) Selects the cache line Word in block \\(\\log_2 K\\) Selects the cell/word within the block Numerical example: direct mapping MM = 4 Gbytes addressable → \\(E = 32\\) bits Cache: \\(Q = 1024\\) lines, \\(K = 64\\) bytes/block Line bits: \\(\\log_2 1024 = 10\\) Word bits: \\(\\log_2 64 = 6\\) Tag bits: \\(32 - 10 - 6 = 16\\) Total blocks: \\(B = 4G / 64 = 64M\\) → \\(\\mathbf{65{,}536}\\) blocks mapped to each line! Aspect Direct mapping Advantage Simple and cheap to implement Disadvantage Block conflict: two blocks in the same line replace each other, even if other lines are free To solve this constant \u0026ldquo;crushing\u0026rdquo; problem of direct mapping, engineers proposed the opposite extreme: giving the block total freedom.\nAssociative Mapping (fully associative) # A block can be placed in any cache line.\nThe figure below shows associative mapping, where MM blocks can be stored in any cache line, completely eliminating block conflicts. However, this requires more complex search logic to locate a specific block.\nExample of associative mapping Division of the address:\nField No. of bits Function Tag \\(E - \\log_2 K\\) Identifies the block (= block number in MM) Word in block \\(\\log_2 K\\) Selects the word within the block To locate a block, the controller compares the tag with all \\(Q\\) lines simultaneously (parallel hardware).\nAspect Associative mapping Advantage Completely eliminates block conflicts Disadvantage Complex and expensive search logic; impractical for large caches Since the parallel search across all lines produces very expensive hardware, the industry had to find a middle ground between the rigidity of direct mapping and the high cost of associative mapping.\nSet-Associative Mapping (set-associative) # A compromise between the two previous ones. The cache is divided into \\(C\\) sets of \\(D\\) lines each:\n$$Q = C \\times D \\qquad\\qquad \\text{set} = \\text{block number} \\bmod C$$The figure below illustrates set-associative mapping, where the cache is divided into sets, and each MM block is mapped to a specific set, but can occupy any line within that set. This reduces block conflicts compared to direct mapping, while keeping the search complexity more manageable than in fully associative mapping.\nExample of set-associative mapping Division of the address:\nField No. of bits Function Tag \\(E - \\log_2 C - \\log_2 K\\) Identifies the block within the set Set no. \\(\\log_2 C\\) Selects the set Word in block \\(\\log_2 K\\) Selects the word The tag is compared only with the \\(D\\) lines of the set, not with the whole cache.\n\u0026#x2197; graph LR B0[\"MM: Block 0\n(0 mod C = 0)\"] BC[\"MM: Block C\n(C mod C = 0)\"] B1[\"MM: Block 1\n(1 mod C = 1)\"] BC1[\"MM: Block C+1\n(C+1 mod C = 1)\"] subgraph S0[\"Set 0: D lines\"] L00[\"Line 0\"] L01[\"Line 1\"] end subgraph S1[\"Set 1: D lines\"] L10[\"Line 0\"] L11[\"Line 1\"] end B0 --\u003e L00 BC --\u003e L01 B1 --\u003e L10 BC1 --\u003e L11 Comparison of the Three Techniques # Technique Flexibility Complexity Cost Block conflict Direct Low (1 fixed line per block) Low Low Frequent Associative Total (any line) High High None Set-assoc. Medium (\\(D\\) lines per set) Medium Medium Reduced Most widely used technique in modern processors Set-associative mapping with \\(D = 2\\), \\(4\\), or \\(8\\) ways dominates current designs. It balances the low cost of direct mapping with the flexibility of fully associative mapping.\nThe Three Types of Miss # With the three mappings on the table, we can classify the misses, and the classification matters, because each type is fought in a different way. The taxonomy is known as the three C\u0026rsquo;s, made canonical by Mark Hill and Alan Jay Smith in the late 1980s:\nType Why it happens How to attack it Compulsory (or cold start) It is the first access to that block; it was never in the cache Unavoidable by definition: it can only be anticipated, with prefetch Capacity The set of data the program uses is larger than the entire cache Larger cache, or rewrite the program to work in smaller pieces Conflict The block fit in the cache, but the mapping forced the eviction of another one that was still useful More associativity The third is the block conflict that appears throughout the direct mapping section, now with a name of its own. And the classification explains the comparison table at once: in fully associative mapping the conflict miss does not exist, because no block is forced to go to a specific place. What remains are the compulsory ones, which nobody avoids, and the capacity ones, which only a larger cache solves.\nLet us see how to compute the address fields for each technique in a concrete example:\nExample: address fields in the three types of mapping A machine can address 4 Gbytes of physical memory with 2-byte cells, organized in 1 Kbyte blocks. The cache holds 2K lines (one block per line). Compute the address fields for the three types of mapping.\nGiven: 4 Gbytes; cell = 2 bytes → \\(N = 2\\text{G} = 2^{31}\\) cells → \\(E = 31\\) bits. Block = 1 Kbyte = 512 cells → \\(B = 2^{31}/2^9 = \u003e 2^{22} = 4\\text{M}\\) blocks. Cache: \\(Q = 2\\text{K} = 2^{11}\\) lines.\nDirect mapping: $$\\underbrace{11}_{\\text{tag}} \\underbrace{11}_{\\text{line}} \\underbrace{9}_{\\text{word}} = 31\\ \\text{bits}$$ line field: \\(Q = 2^{11}\\) → \\(\\mathbf{11}\\) bits; word field: \\(2^9\\) → \\(\\mathbf{9}\\) bits tag field: \\(B/Q = 2^{22}/2^{11} = 2^{11}\\) → \\(\\mathbf{11}\\) bits Fully associative mapping: $$\\underbrace{22}_{\\text{tag}} \\underbrace{9}_{\\text{word}}$$ tag field: \\(B = 2^{22}\\) → \\(\\mathbf{22}\\) bits; word field: \\(\\mathbf{9}\\) bits Set-associative mapping (\\(D = 4\\)):\n\\(C = Q/D = 2^{11}/4 = 2^9\\) sets $$\\underbrace{13}_{\\text{tag}} \\underbrace{9}_{\\text{set}} \\underbrace{9}_{\\text{word}} = 31\\ \\text{bits}$$ set field: \\(\\mathbf{9}\\) bits; tag field: \\(B/C = 2^{22}/2^9 = 2^{13}\\) → \\(\\mathbf{13}\\) bits; word field: \\(\\mathbf{9}\\) bits The interpretation of the fields becomes clearer when we look at what happens on a hit and a miss for a specific address:\nExample: hit and miss for a given address A machine with 32 Gbytes of physical memory has 4-byte cells and a cache with 16 M blocks (1 block per line, 8 cells per block). For address 2 (binary: 0000...010), what characterizes a hit and a miss in each type?\nGiven: \\(N = 8\\text{G} = 2^{33}\\) → \\(E = 33\\) bits. Block = 8 cells → \\(B = 2^{33}/2^3 = 2^{30}\\) blocks. Cache: \\(Q = 2^{24}\\). Word field: \\(2^3\\) → \\(\\mathbf{3}\\) bits.\nIt is worth noting that this cache has \\(2^{24}\\) lines of 32 bytes each, that is, 512 MB, two orders of magnitude above any real L3. The values were chosen so that the powers of 2 come out round; the procedure is identical at any scale.\nDirect mapping: line field: \\(\\mathbf{24}\\) bits; tag: \\(2^{30}/2^{24} = \u003e 2^6\\) → \\(\\mathbf{6}\\) bits. Address 2: 000000 | 000...000 | 010.\nHit: goes to the line given by the 24 middle bits, compares the tag; if equal, direct access. Miss: different tags → whole block transferred from MM. Fully associative: tag field: \\(\\mathbf{30}\\) bits.\nHit: searches all lines in parallel; if any tag matches → hit. Miss: no line has the tag → block transferred. Set-associative (\\(D = 4\\)): \\(C = 2^{24}/4 = 2^{22}\\); set field: \\(\\mathbf{22}\\) bits; tag: \\(2^{30}/2^{22} = 2^8\\) → \\(\\mathbf{8}\\) bits.\nHit: finds the set by the 22 middle bits; compares the tag in the 4 lines. Miss: none of the 4 lines has the tag → block transferred. Replacement Algorithms # When a miss occurs and there is no free line available, the controller has to decide which old block will be removed to make room for the new one.\nThis decision only exists in cases where a choice is possible, that is, in associative mapping and set-associative mapping. In direct mapping, the new block always replaces the block that already occupies the single line compatible with it.\nAlgorithm Strategy Note LRU (Least Recently Used) Removes the block unaccessed for the longest time Tends to work well when there is strong temporal locality FIFO Removes the block loaded the longest ago Simple to implement LFU (Least Frequently Used) Removes the block with the fewest accumulated accesses May preserve old blocks that are no longer useful Random Random choice Simple hardware About LRU LRU is usually a good strategy because it approximates the principle of temporal locality. Still, real implementations often use approximations of LRU, especially in larger caches, to reduce the cost of the control hardware.\nWrite Policies # Reading from the cache is simple: just check whether the block is present. Writing is more delicate, because an update made in the cache can leave Main Memory with an outdated copy. The write policies define how this synchronization will be handled.\nWrite Through # Every write updates the cache and Main Memory at the same time Keeps MM always coherent with the cache Simple to understand and implement Can greatly increase write traffic to MM Write Back # The write updates only the cache The block receives a modification mark (dirty bit) MM is only updated when that block is removed from the cache Reduces write traffic, but requires additional control Write Once, Which Is Not a Write Policy # Textbooks usually present write once as a third policy, on the same level as the two previous ones. The behavior described is this:\nthe first write to a block is done as write through; subsequent writes are treated as write back. But the classification is misleading. Write once was proposed by James Goodman in 1983 as a cache coherence protocol, a different category. The problem it solves is not keeping main memory up to date; it is warning the other caches that that block has changed. The first write goes to MM precisely because it is the bus that the other cores observe: there it works as a signal, not as synchronization.\nAnd it is one of the predecessors of the protocol that dominates current systems, MESI, a name formed from the four states that each cache line can assume: Modified, Exclusive, Shared, and Invalid. That is the term to search for when the subject is coherence, and it is the one that reappears in the last article of the series, when dealing with multiprocessors.\nCache coherence in multicore systems In systems with multiple cores or multiple processors, each core can keep local copies of the same block in its own cache. If one of them modifies that block, the other copies may become outdated. This is the problem of cache coherence.\nWrite policies influence this scenario, but do not solve it alone: what solves it is a coherence protocol implemented in hardware, and in the vast majority of current processors that protocol is MESI or one of its variants.\nThe classic literature, as pointed out in William Stallings\u0026rsquo; book, shows that approximately 15% of all memory operations are writes, since the vast majority of accesses consist of reads to fetch instructions and data. This number has been used as an argument that write through would suffice in most cases, but current practice is subtler, and depends on the level of the hierarchy.\nSome microarchitectures use write through on the L1 toward the L2, like the data L1 of the AMD Bulldozer: the distance between the two is short, a write buffer absorbs the traffic, and the simplicity pays off. The L2 and L3, on the other hand, are typically write back, for a reason that the memory wall explains well. The scarcest resource in the system is DRAM bandwidth, and writing straight to that level would waste bandwidth on values that will still be overwritten in the cache itself before anyone needs them. Accumulating the writes and paying once, when the line is evicted, is what keeps the memory bus free for what matters.\nPhysical Organization of Memory # The 1D and 2D physical organization is an important memory implementation detail, but here it should be understood as a hardware backdrop for memory chips in general, and not as the core of the logical operation of the cache.\nWhether we are talking about Cache or Main Memory, at the lowest hardware level, billions of bits need to be stored in transistors inside silicon chips. The way these circuits are designed so that an address finds its physical cell happens in two main ways:\nType 1D: Linear Selection # The address directly drives one of \\(2^E\\) decoder outputs Conceptually simple Scales poorly for large values of \\(E\\) Type 2D: Row/Column Matrix # The address is split between row and column The cell is selected at the intersection between them Drastically reduces the size of the decoders Example with \\(E = 14\\) bits:\n$$7\\ \\text{row bits} + 7\\ \\text{column bits} \\implies 128 \\times 128 = 16K\\ \\text{cells}$$ \u0026#x2197; graph TD ADDR[\"Address: 14 bits\"] DL[\"Row Decoder\n7 bits → 128 outputs\"] DC[\"Column Decoder\n7 bits → 128 outputs\"] CEL[\"Selected cell\nRow × column intersection\"] ADDR --\u003e|\"bits 13 to 7\"| DL ADDR --\u003e|\"bits 6 to 0\"| DC DL --\u003e CEL DC --\u003e CEL Organization Decoder outputs (\\(E = 14\\)) Hardware efficiency 1D (linear) \\(2^{14} = 16{,}384\\) outputs Low 2D (matrix) \\(2^7 + 2^7 = 256\\) outputs High Conclusion and Next Articles # The cache is a small, fast, expensive component that, when well placed in the memory hierarchy, multiplies the efficiency of the whole system without requiring changes to the software. Each design decision (type of mapping, replacement algorithm, write policy) is a compromise between performance, cost, and implementation complexity.\nFor those who want to go deeper, much deeper, the canonical reference on the subject is What Every Programmer Should Know About Memory, by Ulrich Drepper (2007). It is about a hundred open pages that start more or less where this article ends: real measurements, and the effect of each cache decision on real code.\nIn the next article, we will talk about the CPU. The components that make up the central processing unit, how they are interconnected, and what happens step by step when an instruction is fetched and executed.\nSee you then.\n","date":"2026-04-14","externalUrl":null,"permalink":"/2026/04/cache-memory/","section":"Blog","summary":"With a 95% hit rate, the cache makes the processor see a 4.95 ns memory where DRAM takes 80 ns. We will see why it works (locality), the direct, associative, and set-associative mappings, the replacement algorithms, and the write policies.","title":"Cache Memory: Mapping, Replacement, and Writing","type":"posts"},{"content":"","date":"2026-04-09","externalUrl":null,"permalink":"/tags/dram/","section":"tags","summary":"","title":"DRAM","type":"tags"},{"content":"","date":"2026-04-09","externalUrl":null,"permalink":"/tags/memoria/","section":"tags","summary":"","title":"memória","type":"tags"},{"content":"","date":"2026-04-09","externalUrl":null,"permalink":"/tags/memory/","section":"tags","summary":"","title":"memory","type":"tags"},{"content":" In the introduction to this series, we described what happens in the nanoseconds between pressing Enter and seeing the result of a program. Before any processing can happen, though, there is a prior question: data and instructions must be accessible to the processor. That is where memory comes in.\n\u0026ldquo;Memory\u0026rdquo; is an entire storage architecture designed to balance three forces in permanent tension: speed, capacity and cost. Memory fast enough for the processor (SRAM) is too expensive for large volumes; cheap memory (DRAM, disks) is too slow to keep up with the processor. Computer engineering solved this with the memory hierarchy: using several types at once, each in the role that suits it.\nIn this article you will learn:\nwhat memory does and what its fundamental units are; how the memory hierarchy is organized and why each level exists; the formulas that determine the size of addresses, registers and buses in any memory system; how the CPU and memory communicate through dedicated registers and buses; read and write operations in formal notation; the SRAM, DRAM and ROM technologies and the ECC error correction mechanism. Fundamental concepts # Memory is the component of a computer system responsible for storing the information that the processor (CPU) will manipulate.\nBasic operations # Operation Technical name Description Store Write Record information in memory Retrieve Read Obtain information from memory Storage units # To make things easier before we get into technical jargon, imagine memory as a gigantic warehouse with thousands of lockers (or an immense library). The whole warehouse is your computer\u0026rsquo;s memory.\nThe figure below illustrates this analogy: a warehouse with thousands of lockers, where each locker represents a memory cell, a numbered space of fixed size, ready to hold an exact amount of information.\nAnalogy of memory as a warehouse of numbered lockers. Image source Each individual locker, with room for exactly a specific number of items, is what we call a cell. So that the processor can store and retrieve information quickly without rummaging through the whole warehouse, each locker has a unique number painted on its door: that is the address. When the computer needs to read or write a piece of data, it does not look for the data by what it is, but by the address where it should be, going straight to the right locker.\nWith this image in mind, it is easier to understand the formal units that define this \u0026ldquo;warehouse\u0026rdquo;:\nUnit Definition Bit Basic element; takes only the values 0 or 1 Cell Group of bits handled together; unit of storage and transfer Address Number that uniquely identifies each cell in memory Word Unit of information of the CPU/MM system; represents a number or an instruction Relation between addresses and cells With \\(E\\) address bits, memory can address \\(N = 2^E\\) distinct cells. Therefore: \\(E = \\log_2 N\\).\nMemory is organized as a one-dimensional array of cells, each with a unique address. The processor accesses memory using these addresses, reading or writing data in specific cells. The figure below shows how information is located in memory: the processor supplies an address, the memory finds the corresponding cell and performs the read or write operation.\nLocating information in memory. Memory hierarchy # The memory hierarchy is an organizational structure that classifies the different types of memory into levels, based on their speed, cost and capacity characteristics.\nThe goal of this hierarchy is to optimize system performance, ensuring that the processor has fast access to the most frequently used data, while the less accessed data is stored in slower, cheaper memories. Below, we present the typical levels of the memory hierarchy, from the fastest and most expensive to the slowest and cheapest:\nMemory hierarchy To make this pyramid concrete, the table below details the physical technologies used in each layer, showing how speed and cost behave in practice:\nLevel Typical technology Typical access time Cost/bit Volatility Example Registers Processor internal circuits \\(\\approx 0.3\\) ns Very high Volatile CPU registers L1/L2/L3 cache SRAM \\(\\approx 1\\) ns (L1) to \\(\\approx 12\\) ns (L3) High Volatile Processor internal cache Main memory DRAM \\(\\approx 80\\) ns Medium Volatile RAM Secondary storage Flash / magnetic \\(\\approx 100\\) µs (SSD) to \\(\\approx 10\\) ms (HDD) Low Nonvolatile SSD, NVMe, HDD For teaching purposes, media such as CD-ROM may appear in traditional classifications, but today SSDs and HDDs better represent the secondary storage found in real computers.\nThe times in the third column are orders of magnitude, not specifications: they vary with the microarchitecture, the model and the generation. What matters is the ratio between them, and it becomes more tangible on a human scale. If a register access took one second, an L1 cache access would take about three seconds, L3 about forty seconds, main memory four and a half minutes, an SSD almost four days and a hard disk more than a year. This disparity is what the memory hierarchy exists to hide.\nFor figures measured on real hardware, two sources are worth a visit: 7-cpu.com publishes cache and memory latencies by microarchitecture, in cycles and in nanoseconds; and the interactive page Latency Numbers Every Programmer Should Know shows the full scale (including SSD, disk and network) with its evolution over the years.\nMemory analysis parameters # Access time: interval between the request and the availability of the data Capacity: total amount of information that can be stored Manufacturing technology: semiconductor, magnetic, optical, etc. Persistence: volatile (loses data without power) × nonvolatile Cost: price per byte Although the computer uses all these levels together to balance cost and speed, the real \u0026ldquo;stage\u0026rdquo; where program execution happens is the Main Memory (RAM). So, from now on, we will leave disks and caches aside to focus exclusively on how this main memory is structured inside.\nOrganization of main memory # Main memory can be described as a set of \\(N\\) cells, each with \\(M\\) bits. The address identifies the cell to be accessed, while the word is the CPU\u0026rsquo;s natural unit of processing. These two ideas do not always coincide: a word may occupy one or more cells, depending on the architecture.\nFundamental formulas # $$N = 2^E \\qquad\\qquad E = \\log_2 N \\qquad\\qquad T = N \\times M$$where:\n\\(N\\) = number of cells \\(E\\) = number of address bits (= size of the MAR) \\(M\\) = number of bits per cell (= word size / MDR) \\(T\\) = total number of bits stored in memory Cell, word and transfer unit # In many systems, the cell has 8 bits (1 byte), but that does not mean the processor\u0026rsquo;s word also has 8 bits. The cell is the addressable unit of storage; the word is the CPU\u0026rsquo;s natural unit of processing; and the transfer unit is the number of bits moved in a single read or write operation.\nWord is the number of bits the CPU treats as its natural unit of processing. Historically, this varied a lot; in current general-purpose architectures, it is usually associated with 16, 32 or 64 bits. For example, on a \u0026ldquo;32-bit\u0026rdquo; CPU, the natural width of the integer registers and integer operations is usually 32 bits (4 bytes).\nThe word has important practical implications. Its width often coincides with that of the CPU\u0026rsquo;s integer registers, although that does not mean all the architecture\u0026rsquo;s registers necessarily have this size. Another consequence is alignment: many architectures prefer to store words at addresses that are multiples of their size (for example, 4-byte words at addresses that are multiples of 4), which allows more efficient accesses; misaligned accesses may require multiple cycles or special handling.\nIn addition, the order of the bytes within a word is determined by endianness (little-endian or big-endian). While memory stores bytes, the interpretation of those bytes as a word depends on this convention. Endianness is the convention that defines the order in which the bytes of a value larger than 1 byte are stored in memory. In the little-endian format, the least significant byte comes first; in big-endian, the most significant byte comes first. For example, the hexadecimal value 0x12345678 would be stored as 78 56 34 12 in little-endian and as 12 34 56 78 in big-endian. This does not change the value itself, but it affects how the bytes are interpreted when reading or transmitting binary data.\nFinally, there is a distinction between byte addressing and word addressing: in architectures with byte addressing, each address refers to one byte; in architectures with word addressing, each address refers to an entire word.\nThe transfer unit may also differ from the word. For example, the width of the data bus defines how many bits can be transferred per bus operation, and a cache line defines the transfer unit between levels of the memory hierarchy. In both cases, this choice affects latency, throughput and system design.\nIn practice, it is common for a word to occupy multiple cells:\nWord size Number of 8-bit cells 8 bits 1 cell 16 bits 2 cells 32 bits 4 cells 64 bits 8 cells For this reason, it is best to avoid saying that \u0026ldquo;\\(M\\) is at the same time the size of the cell, of the word and of the MDR\u0026rdquo;, because that only holds in specific architectures.\nMost modern systems adopt the byte as the basic unit of storage.\nCapacity prefixes (powers of 2) # Since the number of cells in a modern RAM easily reaches the billions, we use multiplicative prefixes based on powers of 2 to simplify the mathematical notation:\nPrefix Symbol Value Kilo K \\(2^{10} = 1024\\) Mega M \\(2^{20} \\approx 10^6\\) Giga G \\(2^{30} \\approx 10^9\\) Tera T \\(2^{40} \\approx 10^{12}\\) Peta P \\(2^{50} \\approx 10^{15}\\) Note in the table above that, although the prefixes are similar to those used in the SI, their values are based on powers of 2, not of 10. In the tradition of computer architecture, 1 KB means \\(2^{10} = 1024\\) bytes, and that is the sense in which the prefixes appear throughout this series. But this tradition has a naming problem that is worth knowing about.\n1 kB is a thousand bytes: the binary prefix has its own name Calling \\(2^{10}\\) bytes \u0026ldquo;1 kilobyte\u0026rdquo; was informal practice for decades, and it is the origin of a confusion that persists. In the SI, the prefix k is worth exactly 1,000, so 1 kB is 1,000 bytes. Since the IEC 60027-2 standard (approved in 1998 and published in 1999), powers of 2 have their own prefixes, the binary prefixes kibi (Ki), mebi (Mi) and gibi (Gi): \\(2^{10}\\) bytes are 1 KiB, \\(2^{20}\\) are 1 MiB and \\(2^{30}\\) are 1 GiB.\nThe practical consequence shows up on anyone\u0026rsquo;s screen: a hard drive advertised as \u0026ldquo;1 TB\u0026rdquo; holds \\(10^{12}\\) bytes, which the operating system displays as 931 GB. Nobody was robbed of 69 GB: the manufacturer counted in powers of 10, the system divided by \\(2^{30}\\), and the label the system should display is 931 GiB.\nWhy keep binary K, M and G here, then? Because in the case of addressed memory the power of 2 is not a convention, it is a consequence: with \\(E\\) address bits there are exactly \\(2^E\\) cells. A disk\u0026rsquo;s capacity, on the other hand, has no reason at all to be a power of 2, and that is precisely where the two systems collide.\nNow that we know how to size this memory, a practical problem arises: how does the processor (which is extremely fast) manage to \u0026ldquo;talk\u0026rdquo; to main memory to read and write those billions of bits without getting lost? This is where components dedicated to building that communication bridge come in.\nRegisters and buses # Communication between the CPU and main memory is done by dedicated registers and by buses.\nThe MAR (Memory Address Register) holds the address of the cell to be accessed. The MDR (Memory Data Register) holds the data that will be read from or written to memory. The address bus carries addresses. The data bus carries the data. The control bus carries signals such as read, write and wait. These elements exist to organize and enable the communication between processor and memory in a controlled way.\nSo that the CPU does not have to deal directly with the slowness of memory all the time, it delegates the communication to internal \u0026ldquo;waiting rooms\u0026rdquo;, the registers (MAR and MDR). These registers send and receive data through electronic communication paths called buses.\nThe following figure illustrates the communication model between the Central Processing Unit (CPU) and the Main Memory (MM). This integration is carried out through three structural paths that make up the system bus:\nAddress Bus: connects to the CPU\u0026rsquo;s MAR (Memory Address Register) and carries the exact location of the memory cell to be accessed. Data Bus: connects to the MDR (Memory Data Register) and is the path along which the information (the content itself) travels toward memory (write) or coming from it (read). Control Bus: connects to the CU (Control Unit) and is responsible for transmitting command signals, such as the \u0026ldquo;read\u0026rdquo; or \u0026ldquo;write\u0026rdquo; orders, as well as synchronizing the operation. On the right side, the diagram shows that the Main Memory is not connected directly to the bus wires, but uses a memory controller. This controller acts as an interface that decodes the address and control signals received from the CPU to then allow data to flow to or from the MM.\nInterconnection diagram between the CPU and Main Memory via data, address and control buses. Each component illustrated in the diagram above has a strict responsibility to ensure that information travels without errors. We summarize the function and the capacity of each path and \u0026ldquo;waiting room\u0026rdquo; below:\nComponent Function Size MAR Holds the address of the cell to be accessed \\(E\\) bits MDR Holds the data in transfer (read or write) \\(M\\) bits AB: Address Bus Connects MAR → Controller → MM (unidirectional) \\(E\\) bits DB: Data Bus Connects MDR ↔ MM (bidirectional) \\(M\\) bits CB: Control Bus Control signals: read, write, wait Variable Register Transfer Language (RTL) # RTL is a formal notation for describing data transfer operations between registers and memory.\nNotation Meaning \\((X)\\) Contents of register or memory location \\(X\\) \\((X) \\leftarrow (Y)\\) Copy contents of \\(Y\\) to \\(X\\) \\((MM[MAR])\\) Contents of the cell whose address is stored in the MAR Examples of RTL notation:\n\\((MAR) \\leftarrow (PC)\\): copies the contents of the Program Counter to the MAR \\((MDR) \\leftarrow (MM[MAR])\\): reads from the MM the cell pointed to by the MAR and places it in the MDR \\((MM[MAR]) \\leftarrow (MDR)\\): writes to the MM the contents of the MDR at the position indicated by the MAR Read and write operations # For this section, we will use RTL notation to describe step by step how read and write operations happen in practice, from the moment the CPU decides to access memory until the data is delivered or written to the MM. Look again at the interconnection diagram between the CPU and the MM to follow the steps described below.\nRead # The read operation is a process structured in four steps. The processor says where it wants to read, triggers the command and waits for the information to arrive. Here is the exact flow of how this happens:\nStep RTL operation Description 1 \\((MAR) \\leftarrow \\text{desired address}\\) The address is loaded into the MAR 2 READ signal on the CB The controller interprets the request 3 \\((MDR) \\leftarrow (MM[MAR])\\) The contents of the cell are transferred to the MDR 4 CPU register \\(\\leftarrow (MDR)\\) The data is delivered to the processor The read operation is nondestructive: the data remains in the MM even after being read. The processor can read the same cell several times without changing its contents, which is essential for running programs that depend on stable data in memory.\n\u0026ldquo;Nondestructive\u0026rdquo; holds for the processor, not for the physics From the programmer\u0026rsquo;s point of view, reading changes nothing: the value is still there. At the circuit level, however, reading a DRAM is destructive. Reading a cell discharges the capacitor that held the bit, and the sense amplifier must rewrite the value immediately: a restore that happens on every read, in addition to the periodic refresh we will see in the section on DRAM. The magnetic-core memory that DRAM replaced had the same characteristic. What the model presented here guarantees is that none of this leaks outside the controller: the cell that was read still has the same value when the next access arrives.\nThe figure below illustrates the read process. The value 10011, stored at MM address 0110, is transferred to a CPU register, overwriting the register\u0026rsquo;s previous value (11110) there.\nExample of a read operation: the processor sends an address, and the memory responds with the corresponding data. Write # Whereas in a read the processor only asks for a piece of data, in a write operation it must supply two things at the same time: the destination address and the information to be saved. The logical sequencing goes as follows:\nStep RTL operation Description 1 \\((MAR) \\leftarrow \\text{address}\\) The destination address is loaded into the MAR 2 \\((MDR) \\leftarrow \\text{data}\\) The data to be written is loaded into the MDR 3 WRITE signal on the CB The controller receives the write command 4 \\((MM[MAR]) \\leftarrow (MDR)\\) The data is written to the indicated cell The figure below illustrates the write process. The value 11110 is transferred (a copy) from the CPU to the MM and stored in the cell at address 1000, erasing the previous contents (00110).\nExample of a write operation: the processor supplies an address and a piece of data, and the memory writes the data to the indicated position, overwriting the previous contents. Writing is a destructive operation A write permanently erases the previous contents of the cell with no possibility of recovery. A read, by contrast, is nondestructive: the data remains in the MM after being copied to the MDR.\nSizing in practice # With the formulas \\(N = 2^E\\), \\(E = \\log_2 N\\) and \\(T = N \\times M\\) in hand, we go through a series of variations: each one introduces a new way of presenting the problem or a new set of given quantities.\nExample: Parameters from N and M A MM has \\(N = 2K\\) cells of \\(M = 16\\) bits. Determine \\(E\\) (size of the MAR) and \\(T\\).\n\\(N = 2K = 2 \\times 2^{10} = 2^{11}\\) cells\n\\(E = \\log_2(2^{11}) = 11\\) bits → MAR = 11 bits\n\\(T = N \\times M = 2^{11} \\times 16 = 2^{15} = \\mathbf{32K}\\) bits\nExample 2: Finding the number of cells Given: \\(T = 256K\\) bits, \\(M = 8\\) bits. Determine \\(N\\) and \\(E\\).\n$$N = \\frac{T}{M} = \\frac{2^{18}}{2^3} = 2^{15} = 32K \\text{ cells}$$Since \\(N = 2^{15}\\), we know that \\(\\mathbf{E = 15}\\) bits.\nExample 3: Working with \u0026ldquo;words\u0026rdquo; instead of bits Watch the units! Here the capacity is given in words, not in loose bits. Given: MM with 2K words of 16 bits. Determine \\(E\\), the size of the MDR, the largest address and \\(T\\).\n\\(N = 2K = 2^{11}\\) words → \\(\\mathbf{E = 11}\\) bits (size of the MAR) MDR = 16 bits (= word size, M) Largest address: \\(N - 1 = 2048 - 1 = \\mathbf{2047}\\) \\(T = 2^{11} \\times 16 = 32K\\) bits Example 4: Reading multiple cells per access More modern architectures can read several cells at once to save time. Given: MDR = 32 bits, MAR = 24 bits, 2 cells per access.\nNumber of addressable locations: \\(N = 2^{24} = 16M\\) MDR = 32 bits holds 2 cells → each cell = \\(32 / 2 = 16\\) bits \\(T = 2^{24} \\times 16 = 2^{28} = 256M\\) bits Word = MDR = 32 bits in this architecture: the statement gives the MDR as a reference, but, as we saw above, this equality is not a general rule Example 5: Wide buses and fractional cells An advanced scenario where the total capacity and the buses dictate the rules. Given: AB = 33 bits, DB transfers 4 words, cell = 1/8 of a word, \\(T = 64G\\) bits.\n\\(N = 2^{33} = 8G\\) cells \\(M = T / N = 64G / 8G = 8\\) bits per cell Since 1 cell = 1/8 word → word = \\(8 \\times 8 = \\mathbf{64}\\) bits DB width = \\(4 \\text{ words} \\times 64 = \\mathbf{256}\\) bits Python: computing memory parameters # The code below implements a function memory_params that computes the basic memory parameters from different combinations of inputs. It checks the consistency of the data and returns a dictionary with the computed values.\nimport math from typing import Optional def _is_power_of_two(value: int) -\u0026gt; bool: return value \u0026gt; 0 and (value \u0026amp; (value - 1)) == 0 def memory_params( N: Optional[int] = None, E: Optional[int] = None, M: Optional[int] = None, T: Optional[int] = None, ) -\u0026gt; dict[str, int]: \u0026#34;\u0026#34;\u0026#34; Computes basic memory parameters. Parameters: N: number of cells E: number of address bits M: number of bits per cell T: total capacity in bits Exactly two compatible parameters must be given. \u0026#34;\u0026#34;\u0026#34; provided = {k: v for k, v in {\u0026#34;N\u0026#34;: N, \u0026#34;E\u0026#34;: E, \u0026#34;M\u0026#34;: M, \u0026#34;T\u0026#34;: T}.items() if v is not None} if len(provided) != 2: raise ValueError(\u0026#34;Provide exactly two parameters.\u0026#34;) if N is not None and M is not None: if not _is_power_of_two(N): raise ValueError(\u0026#34;N must be a power of 2.\u0026#34;) E = int(math.log2(N)) T = N * M elif E is not None and M is not None: N = 2 ** E T = N * M elif T is not None and M is not None: if T % M != 0: raise ValueError(\u0026#34;T must be divisible by M.\u0026#34;) N = T // M if not _is_power_of_two(N): raise ValueError(\u0026#34;N must be a power of 2.\u0026#34;) E = int(math.log2(N)) elif N is not None and T is not None: if T % N != 0: raise ValueError(\u0026#34;T must be divisible by N.\u0026#34;) M = T // N if not _is_power_of_two(N): raise ValueError(\u0026#34;N must be a power of 2.\u0026#34;) E = int(math.log2(N)) elif E is not None and T is not None: N = 2 ** E if T % N != 0: raise ValueError(\u0026#34;T must be divisible by N.\u0026#34;) M = T // N else: raise ValueError(\u0026#34;Unsupported parameter combination.\u0026#34;) return {\u0026#34;N\u0026#34;: N, \u0026#34;E\u0026#34;: E, \u0026#34;M\u0026#34;: M, \u0026#34;T\u0026#34;: T} # Example 1: N = 2K = 2^11, M = 16 bits result = memory_params(N=2**11, M=16) print(result) # {\u0026#39;N\u0026#39;: 2048, \u0026#39;E\u0026#39;: 11, \u0026#39;M\u0026#39;: 16, \u0026#39;T\u0026#39;: 32768} # Example 4 from the previous section: MAR = 24 bits, T = 256M bits print(memory_params(E=24, T=256 * 2**20)) # {\u0026#39;N\u0026#39;: 16777216, \u0026#39;E\u0026#39;: 24, \u0026#39;M\u0026#39;: 16, \u0026#39;T\u0026#39;: 268435456} The only combination the function rejects is \\(N\\) with \\(E\\): the two describe the same information (\\(N = 2^E\\)) and neither allows us to get to \\(M\\) or \\(T\\). All the other five are solvable, and the function covers all five.\nSRAM, DRAM and ROM # The main technologies discussed in introductory architecture courses are SRAM, DRAM and ROM.\nSRAM (Static RAM): faster and more expensive; widely used in cache memories. DRAM (Dynamic RAM): slower and cheaper; used as main memory. ROM (Read-Only Memory): nonvolatile memory used to store firmware. The difference between SRAM and DRAM lies in how each bit is stored. SRAM uses more complex circuits per bit, which makes it fast, but expensive and less dense. DRAM, on the other hand, stores bits in capacitors, which must be recharged periodically through refresh. This makes it slower, but much more economical for large capacities.\nThe choice between them depends entirely on where the memory will be installed and what its function in the system will be:\nType Speed Cost Refresh Main application SRAM (Static RAM) Fastest Most expensive Not needed L1/L2 cache DRAM (Dynamic RAM) Slowest Cheapest Needed (periodic) Conventional MM ROM (Read Only Memory) Fast reads Variable Not needed BIOS, firmware After all, why is SRAM so much more expensive and faster, while DRAM requires this refresh? The answer lies in the physics of the thing. SRAM uses a complex circuit (usually with six transistors) for each bit. This makes it ultra-fast and stable, but it takes up a lot of physical space and costs a lot.\nDRAM, in turn, stores each bit in a more ingenious and simple way: as a tiny electric charge in a capacitor. The catch? That capacitor \u0026ldquo;leaks\u0026rdquo; energy over time. So the system needs to read and rewrite (do the refresh of) this memory thousands of times per second so as not to lose the data. It is this need for constant maintenance that makes it a little slower. In return, its simple design allows billions of capacitors to fit in a small space, creating memory modules with gigabytes of capacity at a super affordable cost.\nWhere DRAM comes from, and where \u0026ldquo;core dump\u0026rdquo; comes from The idea of storing a bit in a single transistor and a capacitor is Robert Dennard\u0026rsquo;s, who invented it at IBM in 1966 and patented it in 1968 (patent US 3,387,286, filed in 1967). In 1970, the Intel 1103 made DRAM a commercial product, with 1,024 bits, enough to start retiring the dominant technology until then, magnetic-core memory: tiny ferrite rings threaded with wires, magnetized in one direction or the other to store each bit.\nThat memory died, but it left behind a word that every programmer uses without knowing its origin: when a program crashes and the system dumps the contents of memory to disk, the resulting file is called a core dump, the dump of the cores, the ferrite cores. The name has outlived the technology by half a century.\nROM, firmware, BIOS and UEFI # Didactically, ROM is presented as a nonvolatile memory used to store permanent system startup instructions. In current computers, however, it is more precise to speak of firmware stored in flash memory, normally associated with UEFI, while the term BIOS appears more as a historical or compatibility reference.\nIt is worth abandoning, therefore, the idea that \u0026ldquo;ROM cannot be rewritten\u0026rdquo;. What can be stated precisely is something else: nonvolatile memories used for firmware are not intended for ordinary writing by the CPU during the normal operation of the system, which does not mean absolute immunity to failures or tampering. Their main role is to store startup code and low-level configuration.\nMain applications:\nBIOS (Basic Input/Output System) Bootstrap / IPL (Initial Program Load): program that initializes the system Firmware: microwave oven, video games, automotive electronic fuel injection Memory errors and ECC # Real memories are subject to occasional failures, such as bit flips caused by electrical noise, electromagnetic interference, physical defects or high-energy particles. To reduce the effects of these problems, some systems employ ECC (Error Correction Code).\nECC adds check bits to the data that is stored or transferred, allowing it to:\ndetect certain errors; automatically correct single-bit errors; increase system reliability. This mechanism is especially important in environments where integrity and availability are critical, such as servers, high-reliability workstations and scientific systems.\nThe ECC mechanism detects and corrects these errors:\n\u0026#x2197; %%{init: {'flowchart': {'useMaxWidth': false}}}%% graph TB D[\"M data bits\"] A1[\"Algorithm A\ngenerates K1 check bits\"] GR[\"Writes M + K1 bits to the MM\"] LE[\"Read:\nM bits + K1 stored bits\"] A2[\"Algorithm A\ngenerates K2 check bits\"] CMP{\"K1 = K2?\"} OK[\"No error\nCorrect data\"] ERR[\"Error detected\nand corrected\"] D --\u003e A1 --\u003e GR --\u003e LE --\u003e A2 --\u003e CMP CMP --\u003e|\"Yes\"| OK CMP --\u003e|\"No\"| ERR In the flow above, \u0026ldquo;Algorithm A\u0026rdquo; is a mathematical implementation based on Information Theory. It works by adding extra bits (redundancy or check bits) to the original data before writing it.\nSimple detection (parity bit): the most basic algorithm just counts how many 1 bits there are in the word and adds an extra bit so that the total is always even (even parity). If on reading the number of 1s is odd, the system knows a bit flipped. The catch? It warns that there is an error, but does not know which bit is wrong, which makes correction impossible. Active correction (Hamming code): to be able to correct, the ECC mechanism uses more sophisticated algorithms, such as the Hamming code. It interleaves several parity bits within the same word. If a bit suffers a flip, the combination of errors in the parities points to the exact position of the faulty bit. Knowing which bit is wrong, the memory simply flips it back to the correct value, delivering the information intact to the CPU. The price of safety: ECC in practice If ECC is so good, why doesn\u0026rsquo;t it come in every computer? The reason is the triple cost: space, money and time. For every 64 bits of data, an ECC module needs 8 extra physical bits just for control, which makes manufacturing more expensive. In addition, computing the redundancy on input and output adds a slight latency.\nThat is why standard desktop memories favor speed and low cost (a wrong pixel in a game is not the end of the world). ECC memories, on the other hand, are the standard in servers, data centers and critical systems, where a silently flipped bit can corrupt an entire database or change a financial value.\nThe alleged bit flip in a Super Mario 64 speedrun # As we saw in the previous section, memory is not a shielded, perfect environment. Data traffic on the bus and even storage in the cells are subject to physical interference that causes bit flips (a \\(0\\) that becomes \\(1\\), or vice versa). But what causes this? It can be anything from electromagnetic noise from nearby components, power grid fluctuations, to cosmic rays (high-energy subatomic particles from space that collide with the memory chip).\nMany of you may have heard of bit flips because of a curious episode in the Super Mario 64 speedrunning community. For those who don\u0026rsquo;t know, speedrunning is the practice of completing a game in the shortest possible time, and Super Mario 64 is one of the most popular games for it.\nOne of the most persistent myths in the Super Mario 64 speedrunning community is the claim that a cosmic ray caused an upwarp error (an unexpected vertical teleport) in the Tick Tock Clock level during a run in 2013. Although the phenomenon gained internet fame as an example of space interference in electronic hardware, experts in disassembling the code and hardware of the Nintendo 64 console argue that there is no evidence to support this theory. Instead of ionized particles from space, the most likely technical explanation lies in ordinary physical faults, such as dirty pin connections, poor contact in the cartridge slot or the natural wear of the device\u0026rsquo;s components over decades, although the exact cause was never established.\nThe popularization of this theory as a scientific fact is attributed to a succession of failures in game-specialized journalism, which amplified the hypothesis without checking its technical veracity with the community of TASers (Tool-Assisted Speedrunners, players who use assistance tools). The narrative gained viral strength through social media and citations on far-reaching educational channels, such as Veritasium, without due critical analysis, creating a cycle of misinformation. Today, the case serves as an example of how an attractive but incorrect theory can become truth accepted by the casual public.\nThe video below shows the detailed explanation of the phenomenon, debunking the bit flip myth and presenting the technical evidence that points to more plausible causes. It is worth watching to understand how the speedrunning community and hardware experts reached this conclusion.\nPython: simulating a parity bit # To take the concept out of theory, here is a simplified Python example that implements the logic behind the even parity bit. The goal is to ensure that the final transmitted word always has an even number of 1s.\ndef add_even_parity(bits): \u0026#34;\u0026#34;\u0026#34;Computes and adds the even parity bit to a string of bits.\u0026#34;\u0026#34;\u0026#34; ones_count = bits.count(\u0026#39;1\u0026#39;) parity_bit = \u0026#39;0\u0026#39; if ones_count % 2 == 0 else \u0026#39;1\u0026#39; return bits + parity_bit def check_error(received_data): \u0026#34;\u0026#34;\u0026#34;Checks whether the received data has an error using even parity.\u0026#34;\u0026#34;\u0026#34; ones_count = received_data.count(\u0026#39;1\u0026#39;) if ones_count % 2 != 0: return \u0026#34;Error detected! The data was corrupted.\u0026#34; return \u0026#34;Data apparently intact (no 1-bit errors).\u0026#34; # --- Simulating in practice --- # 1. The CPU wants to write the data \u0026#34;1011001\u0026#34; (it has four \u0026#39;1\u0026#39;s) source_data = \u0026#34;1011001\u0026#34; written_data = add_even_parity(source_data) print(f\u0026#34;Data written to memory: {written_data}\u0026#34;) # Expected result: 10110010 (added a 0 at the end) # 2. Simulate interference: flip a bit in the data (not in the parity bit) # let\u0026#39;s flip the bit at index 5 (0-based) of the data block data = written_data[:-1] # data part without the parity bit parity = written_data[-1] # parity bit i = 5 # index of the bit to flip (0-based) corrupted_data = data[:i] + (\u0026#39;1\u0026#39; if data[i] == \u0026#39;0\u0026#39; else \u0026#39;0\u0026#39;) + data[i+1:] corrupted_word = corrupted_data + parity print(f\u0026#34;Data read with a defect: {corrupted_word}\u0026#34;) # Expected result: 10110110 (now we have five \u0026#39;1\u0026#39;s) # 3. The memory controller performs the check print(f\u0026#34;Read status: {check_error(corrupted_word)}\u0026#34;) # Expected result: \u0026#34;Error detected! The data was corrupted.\u0026#34; Locality of reference: the bridge to the cache # The principles that govern how programs access memory have consequences for system design. Two empirical patterns stand out.\nThe principle of temporal locality observes that if a memory address was accessed recently, there is a high probability that it will be accessed again soon. The control variable of a for loop, for example, is read and incremented on every iteration, and is reused throughout the execution of the loop.\nThe principle of spatial locality observes that if an address was accessed, it is likely that neighboring addresses will also be accessed in sequence. When traversing an array, accessing element a[i] implies a high probability of accessing a[i+1] right after.\nThese two principles are the justification for the existence of cache memory: it keeps recently accessed data in its slots (exploiting temporal locality) and loads entire blocks of contiguous cells on a miss (exploiting spatial locality). Without these patterns observable in real programs, the cache would be useless. With them, the cache can anticipate what the processor will need and have the data ready, without going through the slow DRAM access.\nConclusion and next articles # Main memory is the substrate on which all program execution takes place. We started from its most basic organization (cells, addresses and buses) and arrived at the formulas that size any memory system, passing through read and write operations in RTL notation, the available types of technology and the locality principles that motivate the cache hierarchy.\nIn the next article, we will see how the speed difference between the CPU and DRAM motivated the emergence of the cache, the three mapping models that determine where MM blocks are stored, the replacement algorithms and the write policies.\nSee you then!\n","date":"2026-04-09","externalUrl":null,"permalink":"/2026/04/memory-hierarchy/","section":"Blog","summary":"If a register access took one second, a main memory access would take four and a half minutes, and a hard disk access, more than a year. We will see the memory hierarchy, cells, addresses and buses, the sizing formulas, and SRAM, DRAM, ROM and ECC.","title":"Memory Hierarchy: From Register to Disk","type":"posts"},{"content":"","date":"2026-04-09","externalUrl":null,"permalink":"/tags/rom/","section":"tags","summary":"","title":"ROM","type":"tags"},{"content":"","date":"2026-04-09","externalUrl":null,"permalink":"/tags/sram/","section":"tags","summary":"","title":"SRAM","type":"tags"},{"content":"You press Enter. The terminal displays the result. Three tenths of a second have gone by.\nIn those three hundred milliseconds, a cascade of events unfolded at speeds impossible to visualize intuitively: the processor fetched hundreds of millions of instructions from memory; the cache subsystem tried to anticipate which data would be needed next; the pipeline overlapped the execution of dozens of simultaneous operations; the control unit decoded, one by one, the machine language instructions that correspond to your lines of code, which had previously gone through the phases of lexical analysis, parsing, semantic analysis, and code generation.\nAnd all of this on hardware that, deep down, understands only two states: high or low voltage. Zero or one.\nThis series explores how these layers really work: how memory is organized and addressed, how the processor fetches and executes instructions, what the differences between RISC and CISC are, what a pipeline hazard is, how a decimal number becomes a sequence of bits in memory, how a program in a high-level language reaches the hardware.\nPrerequisites and companion series # This series is self-contained as far as computer architecture is concerned. It does have a sister series, however, From Zero to Float, which covers binary number representation in depth: number bases, two\u0026rsquo;s complement, overflow, and IEEE 754 floating point. If you are not yet familiar with these concepts, I recommend starting there. The article on data representation in this series revisits the main points, but does not replace that reading.\nProgramming in Python is welcome, but not required. All the calculation and reasoning examples are presented mathematically. Python appears as a bonus in some articles for those who want to explore the concepts interactively.\nWhat you will learn in this series # If we could open up a computer and map how its main parts \u0026ldquo;talk\u0026rdquo; to each other, we would see a very well-defined model. On one side, we have the machine\u0026rsquo;s \u0026ldquo;brain\u0026rdquo;: the CPU, which houses the control unit, the calculation units (logic and arithmetic), and the registers. On the other, we have the Main Memory and the devices that interact with the outside world (keyboards, printers, monitors). So that the CPU does not need an individual cable for each part, everything is connected by a main communication path called a bus, as the following diagram illustrates:\nBasic organization of a computer (Von Neumann model), highlighting the communication between the CPU, the Main Memory, and the Input/Output Devices through the central bus. Understanding this organization of a computer is the central goal of this series of articles. By the end, I hope the reader will have a deep understanding of every aspect presented in the figure.\nVon Neumann and Harvard The model in the figure is the Von Neumann one: instructions and data live in the same memory and travel through the same bus. There is an alternative, the Harvard architecture, in which instructions and data have separate memories and buses. It is worth keeping the distinction in mind, because it reappears twice in the series: when the cache splits into one part for instructions and another for data, and when the pipeline stalls because two stages want to access the same memory in the same cycle. Both cases are a consequence of instructions and data sharing the same path, and modern processors solve it with a middle ground called modified Harvard.\nThe series is organized into 13 thematic articles, in addition to this introduction. Each article is dedicated to a specific aspect of computer architecture, but all of them are interconnected to form a complete picture of how a machine works internally. The topics in the table below link directly to each article:\n# Topic Topics covered 0 Introduction This very article 1 Boolean Algebra Logic gates, postulates and theorems, De Morgan\u0026rsquo;s laws, universality of NAND and NOR, and algebraic simplification 2 Simplifying Logic Functions Minterms and maxterms, SOP and POS forms, Karnaugh map, and don\u0026rsquo;t-care conditions 3 Memory Hierarchy Cells, addresses, MAR/MDR, SRAM, DRAM, ROM, ECC, and sizing 4 Cache Memory Locality, direct/associative/set-associative mappings, replacement, and writes 5 CPU: Components and Instruction Cycle ALU, CU, registers, IR, PC, and the fetch-decode-execute cycle 6 CPU: The Hypothetical Processor 12-bit instruction set, step-by-step execution, and hypothetical assembly 7 Pipeline and Buses Stages, data/control/structural hazards, and hardwired × microprogrammed CU 8 Data Representation ASCII, Unicode, two\u0026rsquo;s complement, shifts, and IEEE 754 9 Instruction Representation ISA, formats with 0 to 3 operands, and the 7 addressing modes 10 Program Execution Evolution of languages, compiler phases, assembler, linker, and loader 11 Python and the Path from Code to Hardware (bonus) CPython, bytecode, VM, and differences between interpreted and compiled languages 12 Input and Output Polling, interrupts, DMA, and the I/O overhead calculation 13 Advanced Architectures RISC × CISC, Flynn\u0026rsquo;s taxonomy, SMP, NUMA, clusters, and multicore About the methodology # Each article starts from precise definitions, presents the fundamental formulas and models, works through theoretical and numerical examples step by step, and ends by connecting the topic to the next article.\nThe assembly code examples follow closely the way they appear in academic references, with occasional adaptations for didactic reasons: an address adjusted here, a comment added there, a name changed to keep consistency with the rest of the series. The closeness to the original material is intentional: you will be seeing essentially the same assembly that appears in classic computer architecture textbooks.\nAbout the origin of the series and my background # I have a bachelor\u0026rsquo;s degree in chemistry and later did a master\u0026rsquo;s in chemistry. However, I have always had a strong affinity for computing and electronic equipment in general. So much so that, while still an undergraduate doing undergraduate research, I became responsible for the maintenance and management of the laboratory equipment and its respective computers. Besides that, whenever I could automate something with code, even if it was just an Excel spreadsheet, I automated it.\nSo, naturally, my career evolved toward something related to programming. For example, I was once a chemistry teacher at a federal institute, and in my classes I used interactive materials that I developed myself. I also advised students on projects involving programming. And today I am a data scientist, having worked on several projects for large national and international companies.\nI learned programming on my own, from books, documentation, and material on the internet. Until, because I enjoy studying so much, I decided to start a new undergraduate degree. Honestly, I don\u0026rsquo;t even know if I will see the program through to the end, because of time and work. I started because I like to study, and I was already studying on my own various undergraduate and graduate computing materials. If I think it is worth it, I will continue; otherwise I will leave the program and keep going on my own, as I have always done. But, in any case, this series is based on one of the program\u0026rsquo;s courses. Let\u0026rsquo;s talk a bit about the program and the course that gave rise to the articles in this series.\nAssociate Degree in Computer Systems # This series of articles was inspired by the materials of the course Computer Organization in the Tecnólogo (associate degree) program in Computer Systems offered by CEDERJ.\nIn case you are not from Rio de Janeiro or do not know the project, CEDERJ (Centro de Educação Superior a Distância do Estado do Rio de Janeiro, the Rio de Janeiro State Distance Higher Education Center) is a consortium that brings together the main public universities of the state (such as UFF, UFRJ, UERJ, among others) to offer higher education. In the case of this program, the diploma is issued by UFF. The program is blended: students study through a platform with dedicated teaching materials and online tutoring support, but the official exams take place in person at the regional centers spread across the state.\nIf you have doubts about what a Tecnólogo program is, know that it is a higher-education degree, exactly like a bachelor\u0026rsquo;s or a teaching degree. The main difference lies in focus and duration: while a bachelor\u0026rsquo;s in Computer Science can last four or five years with an extensive theoretical load, the Tecnólogo is leaner (it usually lasts about three years) and has a curriculum that is theoretically more oriented toward practice and toward the real, immediate demands of the technology job market. Emphasis on \u0026ldquo;theoretically\u0026rdquo;, but that is talk for another article.\nSomeday I will write more about the program, whether it is worth it, how it is organized, and other points. Here, it is worth commenting a little on the course and also making some clarifications.\nHow the series was developed # A few very important points:\nI have no connection to CEDERJ other than as a student. I do not know the course\u0026rsquo;s professors (remember that the program is delivered through an online platform with pre-recorded videos) or the tutors. The articles grew out of personal notes I took while studying for the course. Later I complemented them with other things I found interesting that were not covered in the course. And I adapted them to fit the form and language of the other articles here at Ciência Programada. The articles have no pretension whatsoever of replacing the course materials. In fact, I wrote them to be read by anyone interested in the topic, even someone who does not know what CEDERJ is or does not even read this introductory article. Again, the articles are completely independent of the material that served as inspiration. The articles cover several topics not covered in the course that I found interesting to add. For example, since I work with Python, I added several examples in that language that are not original to the course. I included them because I like the language and thought it would be nice to add. The course material is very dated. So I made an effort to update it and complement it with more recent information whenever possible. The course materials served only as inspiration and reference. In order not to violate any copyrights of the professors, the institution, or the books the professors may have used to produce the material, I did not copy any passage, however small, from the slides. I also did not directly use any figure from the slides. Regarding the figures that appear in the articles, I either recreated them using Draw.io or Mermaid, or they were created from scratch by me. Some were also created using AI image generators, such as Gemini, and in those cases you can tell by the watermark that such tools usually leave. The ones I recreated have additions that I found interesting compared to the original figures in the course materials. Regarding the examples that appear in the articles. Some are inspired by examples from the course materials, whether from the slides or from assignments, but were adapted and modified to fit better the form and language of the articles here on the site. Some others are adapted from the books cited below. And there are examples that are original, created by me, with no relation whatsoever to the course materials. In fact, the course material is quite simple and old. And I am not criticizing the professors who originally developed the material. If anyone deserves criticism, it is the institution CEDERJ, which does not invest in updating the material. That is precisely why I went looking for other sources while studying, since the material did not satisfy my curiosity and is quite outdated, especially in format.\nFor those who are curious, here are the course slides. And also the distance assignments from recent years and the exams from recent years. All of these links are from a Google Drive maintained by the program\u0026rsquo;s Student Council, of which I am not a member and with which I have no involvement.\nThe same Student Council uploaded the videos available on the study platform to YouTube, in this playlist. The reason is the platform\u0026rsquo;s instability, which constantly harms students. By staying on YouTube, the chance of being offline drops drastically.\nIf any of these links goes offline The links above point to material hosted by third parties (Google Drive folders and a YouTube playlist), maintained by the program\u0026rsquo;s Student Council. I have no control over either of them, and content like this tends to vanish without notice. If you find a dead link, leave a comment letting me know: I will try to find another way to share the material.\nAs you can see, the videos were recorded a long time ago, almost 20 years. The low image quality is not because of YouTube, it is exactly like that on the platform. Unfortunately, that is what CEDERJ provides. And it is not only in this course that the situation is like this. Again, I am not criticizing the professors who recorded this years ago with the resources and technologies available at the time. I believe not even they thought back then that the material would be used almost two decades later. What I criticize is the institution that does not invest in updating the material, whether in format or in content. I have more than a decade of teaching experience, and I know when a deficiency in material is due to a lack of investment by the institution or a lack of effort by the teacher. And, in this case, it is clearly the institution that does not invest.\nAnyway, someday I will write more about CEDERJ\u0026rsquo;s problems (and also some of the positives). The goal here is just to make clear that the articles, however much they were initially inspired by the course material, do not contain copies of passages from the materials. In fact, I imagine that anyone who reads the articles will notice that they have content, form, and approach quite different from the slides and videos.\nFinally, the course recommends the book \u0026ldquo;Introdução à Organização de Computadores\u0026rdquo; by Mário A. Monteiro, published by LTC. As far as I could determine, the latest edition of this book is from 2007\u0026hellip; The book is good, but I believe a more up-to-date one would be more interesting. I ended up studying more from \u0026ldquo;Computer Organization and Architecture\u0026rdquo; by William Stallings, which has more recent editions and a teaching approach that I found better than that of Monteiro\u0026rsquo;s book.\nNow enjoy the series # That\u0026rsquo;s it for this article. Unlike other series here on the site, I felt this one needed a more detailed introduction before getting straight into the content. There is a whole context that, for more demanding readers, can make a difference. But, overall, I think the series can easily be understood and enjoyed without the personal context covered here.\nFor those reading on the day this article was published, the first article in the series will be available tomorrow. See you then!\n","date":"2026-04-08","externalUrl":null,"permalink":"/2026/04/inside-the-computer/","section":"Blog","summary":"You press Enter and the result shows up three tenths of a second later. In that interval, the processor fetched hundreds of millions of instructions from memory. This series of 13 articles explains how: memory, cache, CPU, pipeline, data representation, and program execution.","title":"Inside the Computer: How Machines Really Work","type":"posts"},{"content":"Um guia definitivo sobre representação numérica: dos Inteiros ao Ponto Flutuante. Entenda como o computador armazena dados, domine o padrão IEEE 754 e aprenda a evitar as armadilhas matemáticas mais comuns, como Overflow, Underflow e erros de precisão.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2026-04-03","externalUrl":null,"permalink":"/series/do-zero-ao-float/","section":"Séries","summary":"","title":"Do Zero ao Float","type":"series"},{"content":"A definitive guide to number representation: from Integers to Floating Point. Understand how the computer stores data, master the IEEE 754 standard and learn to avoid the most common mathematical pitfalls, such as Overflow, Underflow and precision errors.\nThe articles are listed below, in the order they were written.\n","date":"2026-04-03","externalUrl":null,"permalink":"/series/from-zero-to-float/","section":"Series","summary":"","title":"From Zero to Float","type":"series"},{"content":" In the previous articles, we studied integers. We learned that the computer uses zeros and ones to count, that it uses Two\u0026rsquo;s Complement to represent negatives and that it has clear memory limits (Overflow).\nWe also had a brief encounter with binary fractions: the digits to the right of the point have weights \\(2^{-1}, 2^{-2}, 2^{-3}\\dots\\), so that \\(10.101_2\\) is worth \\(2 + 0.5 + 0.125 = 2.625\\) in decimal. But so far we have done these conversions on paper, assuming the point was \u0026ldquo;fixed\u0026rdquo; somewhere.\nBut how does the hardware, with its memory modules limited to 32 or 64 bits, store real numbers? How does it hold, in the same memory structure, the distance between two galaxies (\\(2.5 \\times 10^{19}\\) km) and the diameter of an atom (\\(1 \\times 10^{-10}\\) m)?\nThe answer is to let the point float. Welcome to the IEEE 754 standard, the universal rule that dictates how computers handle real numbers (Floating Point) and the reason why, sometimes, \\(0.1 + 0.2\\) is not exactly \\(0.3\\).\nThe Scale Problem: Why Not Use Fixed Point? # Imagine you have 32 bits to store a number. The most naive approach would be to split that space in half: we would use 16 bits for the integer part and 16 bits for the fractional part. This is called Fixed Point.\nFor many financial applications, this works well (after all, cents only need two decimal places). But for science and engineering, it is a disaster.\nThe Giant Problem: With 16 integer bits, the largest number we can represent is \\(2^{16} - 1 = 65535\\). That is not even enough to count the population of a small town, let alone astronomical distances. The Microscopic Problem: With 16 fraction bits, the smallest difference we can measure is \\(2^{-16}\\) (approx. \\(0.000015\\)). That is useless for measuring quantities in quantum physics. If we increase the integer part, we lose precision in the fraction. If we increase the fraction, we lose range in the integers. The blanket is too short.\nThe Solution: Binary Scientific Notation # To solve this, computer engineers adopted the same strategy as scientists: Scientific Notation.\nIn decimal, we do not write \\(0.000000123\\). We write \\(1.23 \\times 10^{-7}\\). Note that the position of the decimal point \u0026ldquo;floats\u0026rdquo; depending on the exponent.\nThe computer does the same thing, but in binary. It stores numbers in the format:\n\\[ \\pm 1.m \\times 2^e \\]Where:\n\\(\\pm\\): is the sign (positive or negative). \\(m\\): is the significand (the \u0026ldquo;body\u0026rdquo; of the number, e.g., \\(1.01101...\\)). \\(e\\): is the exponent (which moves the point to the left or right). By separating the value from the scale, we can represent colossally large and infinitesimally small numbers using the same number of bits.\nWhy Base 2 and not Base 10 or 16? In the early days of computing (as in the IBM System/370 mainframes), it was common to use Base 16 (Hexadecimal) for floating point. The idea was to increase the range of the exponent.\nHowever, the IEEE 754 standard settled on Base 2. According to David Goldberg\u0026rsquo;s classic paper, \u0026ldquo;What Every Computer Scientist Should Know About Floating-Point Arithmetic\u0026rdquo;, the choice of Base 2 is mathematically superior because of the concept of Wobble (Oscillation).\nIn Base 16, the first significant digit can be 0001 (1) or 1111 (15). This means we can waste up to 3 bits of precision just to represent the leading digit (e.g., 0001 has 3 useless zeros on the left).\nIn Base 2, on the other hand, the first digit is always 1. This maximizes the effective precision and ensures that the relative error varies far less (smaller wobble) than in larger bases.\nFun fact: IBM\u0026rsquo;s hexadecimal format did not die with the System/370. It is still available on z/Architecture mainframes today, for compatibility with code written decades ago, alongside IEEE 754, which is the format used for anything new.\nThe Anatomy of Floating Point (And Why We Chose 32 Bits) # Before we dissect the bits, we need a bit of context. Until the early 1980s, each computer manufacturer (IBM, Cray, DEC) had its own format for real numbers. Code that ran on one mainframe could give different results on another. It was numerical chaos.\nTo solve this, the IEEE 754 standard emerged in 1985, unifying the mathematical language of processors.\nThe standard has an architect with a name and a surname: William Kahan, a Canadian mathematician who was a key consultant on the arithmetic design of the Intel 8087 coprocessor (1980) and turned that design into the basis of the standard that the whole industry would adopt five years later. Kahan received the 1989 Turing Award for his fundamental contributions to numerical analysis, among which is the format dissected in the next pages. This standard defines several number \u0026ldquo;sizes,\u0026rdquo; suited to different needs:\nHalf Precision (16 bits): Widely used today on graphics cards (GPUs) for Artificial Intelligence and Deep Learning, where speed matters more than extreme precision. Single Precision (32 bits): The classic float of the C language. It is the standard for 3D game geometry, digital audio and graphics processing. Double Precision (64 bits): The default float in Python, JavaScript and Ruby. It is essential for scientific simulations and civil engineering, where the accumulated error of 32 bits would be dangerous. Quadruple Precision (128 bits): Used in extreme niches, such as long-term orbital dynamics, ill-conditioned linear algebra and computational chemistry, cases in which the accumulated error of 64 bits already compromises the result. (Cryptography, despite its reputation, is not on this list: it works with exact arbitrary-precision integers, not with floating point.) Our Choice: The 32-Bit Model # In this article, we will focus on the Single Precision (32 bits) structure. \u0026ldquo;But if I use Python (64 bits), why learn the 32-bit one?\u0026rdquo;\nThe answer is didactic: the logic is identical. All of them work by dividing memory into three fields (Sign, Exponent, Significand). The only difference is the number of bits in each field.\nWe chose 32 bits because it is a good balance: complex enough to show all the real problems (such as rounding errors), but short enough that we can write the binaries on screen without filling 3 lines with zeros and ones.\nComparison: 32 bits vs 64 bits # Although the logical structure is the same, the field sizes change to accommodate greater precision and range in the 64-bit standard (used by Python/JS).\nFeature Single Precision (32 bits) Double Precision (64 bits) Common Use C float, GPUs, Games, Audio Python float, JS Number, C double Total Size 32 bits 64 bits Sign 1 bit 1 bit Exponent 8 bits 11 bits Significand 23 bits 52 bits Bias 127 1023 Decimal Precision ~7 significant digits ~15-17 significant digits Range \\( \\approx 10^{-38} \\) to \\( 10^{38} \\) \\( \\approx 10^{-308} \\) to \\( 10^{308} \\) Python uses Double Remember: when you create x = 1.5 in Python, you are using the right-hand column (64 bits). This means you have much more precision than in the manual 32-bit examples we will work through below, but the rules of rounding error and structure are identical.\nOpening the Hood # Now imagine a 32-bit block of memory. Unlike integers (int), where all the bits work together summing powers of 2 to form a single value, here the memory is sliced into three fields with completely distinct functions. It is as if we had three mini-variables packed inside a single one.\nThe official IEEE 754 split for 32 bits is:\nField Size Bits (Position) What does it do? Sign 1 bit 31 Defines whether it is positive (+) or negative (-). Exponent 8 bits 30 to 23 Defines the magnitude (scale). Significand 23 bits 22 to 0 Defines the precision (digits). Visually, the 32 bits are laid out like this:\n[S] [EEEEEEEE] [MMMMMMMMMMMMMMMMMMMMMMM] 1 8 23 Or, in even more detail, as in the following figure. The concepts of normalization and special values will be explained later in the article.\nStructure of the 32-bit IEEE 754 floating point. Some concepts will be explained later. Let us dissect each part of this \u0026ldquo;ruler.\u0026rdquo;\nThe Sign Bit (1 bit) # The leftmost bit (MSB - Most Significant Bit) owns the sign.\n0 indicates a Positive number. 1 indicates a Negative number. Difference: Integers vs. Float Remember the three schemes that competed for the representation of signed integers, back in the article on two\u0026rsquo;s complement? Floating point chose precisely the one that lost that contest.\nFor integers, we use Two\u0026rsquo;s Complement, in which the sign bit has a \u0026ldquo;negative weight\u0026rdquo; and changes the arithmetic of the integer. In Floating Point, the system is Sign-Magnitude: the sign bit is just a flag (an indicator), separate from the magnitude. If you flip only that bit, the magnitude stays identical and the number goes from \\(+x\\) to \\(-x\\). Changing the sign costs the flip of a single bit.\nAnd the flaw that condemned sign-magnitude for integers (the existence of two zeros) becomes a deliberate feature here. We will come back to this point in the section on special values.\nThe Exponent (8 bits) # The next 8 bits define the Scale of the number. This is where the computer decides where the point will be.\nThink of it as the exponent of scientific notation (\\(10^5\\) or \\(10^{-3}\\)). Since there are 8 bits, we can have values from \\(0\\) to \\(255\\). This allows us to represent gigantic magnitudes (up to approx. \\(3.4 \\times 10^{38}\\)) or tiny ones (down to \\(1.18 \\times 10^{-38}\\), among normal numbers). About the exponent The value stored here is not the \u0026ldquo;real\u0026rdquo; exponent of mathematics, but a \u0026ldquo;biased\u0026rdquo; value. We will see this technical trick later.\nThe Significand (23 bits) # The remaining 23 bits (on the right) are the Significand (or Mantissa). This is where Precision lives.\nThe Significand contains the significant digits of the number. While the Exponent says \u0026ldquo;how big\u0026rdquo; the number is, the Significand says \u0026ldquo;how exact\u0026rdquo; it is. If you need to tell \\(1.000001\\) apart from \\(1.000002\\), it is the Significand that must have enough bits to hold that difference. The Hardware Formula # When the computer reads these 32 bits, it does not see the number all at once. It feeds the pieces into the following equation to reconstruct the real value:\n$$ \\text{Value} = (-1)^{\\text{Sign}} \\times (1.\\text{Significand}) \\times 2^{\\text{Exponent} - \\text{Bias}} $$It may look intimidating, but it is just the mathematical version of saying: \u0026ldquo;Take the sign, multiply by the body of the number (1 point something) and shift the point that many places to the left or right.\u0026rdquo;\nSemantics: The Value of the Bits Changes The hardest concept to accept for those coming from Integers is that the value of a bit in Floating Point depends on its neighbor (the Exponent).\nIn a 32-bit integer, the bit at position 0 is always worth \\(1\\) unit. The bit at position 5 is always worth \\(32\\) units. In a float, the last bit of the Significand can be worth \\(0.0000001\\) if the Exponent is small, or it can be worth a million if the Exponent is huge. That is why we say that floats have relative precision, not absolute. The \u0026ldquo;size\u0026rdquo; of the bit changes with the size of the number.\nRelative precision is one of those ideas that a picture settles in seconds. Below, the representable values of a reduced format (3-bit significand, to fit on the page) drawn on the number line. Notice that the number of marks between two powers of 2 is always the same; what changes is the size of the step, which doubles in each range:\nThe same number of marks in each range, an ever larger step: that is what relative precision means. And the exact 0.1 falls right in a gap. The bottom panel is a preview of the topic that closes this article: since the marks do not cover the line, most numbers (including \\(0.1\\)) fall in the gap between two of them. We will return to it.\nThe \u0026ldquo;Hidden Bit\u0026rdquo; (Implicit Bit) Trick and Normalization # If you look at the 32-bit structure, you will see that we reserved 23 bits for the Significand. That gives us a certain precision. But the engineers who designed the IEEE 754 standard realized they could gain an extra bit \u0026ldquo;for free\u0026rdquo; by using a mathematical property of the binary system.\nTo understand this, we need to talk about Normalization.\nWhat Is a Normalized Number? # In decimal scientific notation, we can write the number 300 in several ways:\n\\(300 \\times 10^0\\) \\(30.0 \\times 10^1\\) \\(0.3 \\times 10^3\\) Although mathematically equal, this mess gets in the computer\u0026rsquo;s way (how do you compare numbers if they have different formats?). So there is a rule: scientific notation is Normalized when there is exactly one nonzero digit before the point. In the case above, the normalized form is \\(3.0 \\times 10^2\\).\nNow, on to Binary. In binary, the only digits available are \\(0\\) and \\(1\\). If we apply the normalization rule (\u0026ldquo;the first digit cannot be zero\u0026rdquo;), what is the only option left? The number 1.\nThis means that, in any normalized binary number (gigantic or microscopic), the form will always be:\n\\[ 1.\\text{something} \\times 2^{\\text{exponent}} \\]Examples:\n\\(1.001 \\times 2^5\\) \\(1.1101 \\times 2^{-3}\\) An Important Caveat The \u0026ldquo;Implicit 1 Bit\u0026rdquo; rule holds for the vast majority of numbers, called Normalized Numbers. However, there is an important exception for extremely tiny numbers (close to zero), where this rule is switched off to avoid calculation errors. We call these numbers Subnormal Numbers, and they will get an exclusive section further on. For now, assume the rule always holds.\nThe Engineering Insight: The Ghost Bit # Here comes the standard\u0026rsquo;s trick. If every normalized number necessarily starts with \\(1.\\), why waste a precious bit of memory to store that \u0026ldquo;1\u0026rdquo;? It is redundant. The computer already knows it is there!\nIEEE 754 discards that leading \u0026ldquo;1\u0026rdquo; before writing to memory and stores only what comes after the point (the fractional part).\nWhen the processor reads the number back to do a calculation, it puts that bit back, automatically placing it in front of the significand. This is the so-called Hidden Bit (or Implicit Bit).\nThe Real Gain: Although we have only 23 physical bits in memory for the Significand, the effective precision of the system is 24 bits (23 stored + 1 implicit). This halves the relative error of calculations without costing any extra hardware for storage.\nThe Importance of Uniqueness Besides saving space, normalization solves a critical architectural problem: the Uniqueness of Representation.\nImagine if the computer allowed storing the same value as \\(1.0 \\times 2^1\\) and \\(0.1 \\times 2^2\\). To check whether A == B, the processor would have to perform complex mathematical juggling to align the exponents before comparing.\nBy forcing numbers to be normalized, we guarantee that each real number (within the normalized range) has only one standard binary representation. This avoids the ambiguity of having the same number written in different ways in memory, which makes the hardware\u0026rsquo;s life much easier. Note: The exceptions to this uniqueness are Zero (which has a sign) and NaN.\nHowever, this rule raises an obvious question: \u0026ldquo;If the number always starts with 1, how do we represent Zero?\u0026rdquo; The answer is that Zero is a special exception to the rules, which we will see in the section on \u0026ldquo;Special Values.\u0026rdquo; For now, assume every number has that invisible \u0026ldquo;1.\u0026rdquo; in front.\nThe Exponent and the Bias # We already know how to represent the sign (1 bit) and the body of the number (Significand). Now we need to settle the scale.\nIn scientific notation, the exponent defines whether the number is gigantic (\\(2^{100}\\)) or microscopic (\\(2^{-100}\\)). Since we have 8 bits reserved for the exponent, we can represent \\(256\\) different values.\nThe obvious problem is: we need negative numbers. Without negative exponents, we could only represent numbers greater than 1. We could not represent \\(0.5\\) (\\(2^{-1}\\)) or the size of an electron.\nIn the article on integers, we learned that the standard solution for storing negatives is Two\u0026rsquo;s Complement. However, the IEEE 754 standard does not use Two\u0026rsquo;s Complement for the exponent. It uses a system called Biased Exponent.\nThe Mathematics of the Bias # Instead of sacrificing a bit for the sign, the standard defines a \u0026ldquo;bias\u0026rdquo; value that is added to the real exponent before storing it in memory.\nFor single precision (32 bits), the bias is 127.\nThe storage formula is: $$ E_{\\text{stored}} = E_{\\text{real}} + 127 $$This turns the range of exponents into a scale of positive integers without sign (Unsigned):\nIf you want a Zero exponent (\\(2^0\\)): store \\(0 + 127 = \\mathbf{127}\\) (\\(01111111_2\\)). If you want a Positive exponent (\\(2^{10}\\)): store \\(10 + 127 = \\mathbf{137}\\). If you want a Negative exponent (\\(2^{-10}\\)): store \\(-10 + 127 = \\mathbf{117}\\). With this, the stored exponent ranges from \\(1\\) to \\(254\\) (remember that \\(0\\) and \\(255\\) are reserved for special cases, as we will see later).\nWhy Complicate Things? # You may be wondering: \u0026ldquo;Why not use Two\u0026rsquo;s Complement, which the computer already does natively?\u0026rdquo;\nThe answer lies in comparison speed. Computers frequently need to answer: \u0026ldquo;Is number A greater than number B?\u0026rdquo;.\nIf we used Two\u0026rsquo;s Complement, numbers with negative exponents (starting with bit 1) would look arithmetically \u0026ldquo;greater\u0026rdquo; than numbers with positive exponents (starting with bit 0) if they were read as ordinary integers.\nThe Lexicographic Advantage Using the Bias preserves the lexicographic order of floating-point numbers.\nThanks to the Bias, the exponent of a small number (e.g., \\(2^{-10} \\to 117\\)) yields a smaller bit sequence than the exponent of a large number (e.g., \\(2^{10} \\to 137\\)).\nThe Practical Result: This allows comparing two positive Floating Point numbers using the same simple, fast circuits that compare Unsigned Integers. For negative numbers, the logic needs only a simple sign adjustment, but the structure of the exponent still makes ordering easier.\nHands On: Converting a Number (-12.5) # Theory is good, but it is in manual conversion that we really understand how the gears fit together. Let us convert the decimal number -12.5 to the 32-bit IEEE 754 standard, step by step.\nOur goal is to fill in the three fields: [Sign] [Exponent] [Significand].\nStep 1: The Sign # The number is negative (-12.5). So, the Sign Bit is 1.\nStep 2: \u0026ldquo;Raw\u0026rdquo; Binary # First, we forget the sign and convert the magnitude (\\(12.5\\)) to binary using the Fixed Point method we already know.\nInteger Part (12): \\(1100_2\\) Fractional Part (0.5): \\(0.1_2\\) (since \\(1 \\times 2^{-1} = 0.5\\)) Putting the parts together: \\[ 12.5_{10} = 1100.1_2 \\] Step 3: Normalization (Scientific Notation) # Now we need to move the point to the left until only a single digit \\(1\\) remains before it. To turn \\(1100.1\\) into \\(1.1001\\), we need to move the point 3 places to the left.\n\\[ 1100.1 = 1.1001 \\times 2^3 \\]From here we extract two pieces of information:\nReal Exponent: \\(3\\) Raw Significand: \\(1.1001\\) Step 4: The Biased Exponent # The real exponent is \\(3\\). But remember the section on the bias: we do not store the \\(3\\). We store \\(3 + \\text{Bias}\\). For single precision, the bias is \\(127\\).\n\\[ \\text{Stored Exponent} = 3 + 127 = 130 \\]Now we convert \\(130\\) to binary (8 bits). We know \\(128\\) is \\(10000000\\), so \\(130\\) (\\(128 + 2\\)) is: 10000010\nStep 5: The Final Significand (The Hidden Bit) # We look at the normalized significand from Step 3: \\(\\mathbf{1}.1001\\). Remember the hidden bit: the leading \u0026ldquo;1.\u0026rdquo; is implicit (Hidden Bit). We discard it and store only what comes after the point.\nBits to store: 1001 Since we have 23 bits of space, we fill the rest with zeros on the right. Stored Significand: 10010000000000000000000\nStep 6: Final Assembly # Now we just paste the three parts side by side:\nSign (1 bit): 1 Exponent (8 bits): 10000010 Significand (23 bits): 10010000000000000000000 The 32-bit binary number is: 1 10000010 10010000000000000000000\nBonus Step: Hexadecimal # Reading 32 bits is painful. Programmers prefer Hexadecimal. Let us group the bits in fours and convert:\n$$ \\begin{array}{c c c c c c c c} 1100 \u0026 0001 \u0026 0100 \u0026 1000 \u0026 0000 \u0026 0000 \u0026 0000 \u0026 0000 \\\\ \\downarrow \u0026 \\downarrow \u0026 \\downarrow \u0026 \\downarrow \u0026 \\downarrow \u0026 \\downarrow \u0026 \\downarrow \u0026 \\downarrow \\\\ \\mathbf{C} \u0026 \\mathbf{1} \u0026 \\mathbf{4} \u0026 \\mathbf{8} \u0026 \\mathbf{0} \u0026 \\mathbf{0} \u0026 \\mathbf{0} \u0026 \\mathbf{0} \\end{array} $$Therefore, if you inspect your computer\u0026rsquo;s memory when it stores -12.5 in a float, you will find the Hexadecimal value 0xC1480000.\nValidation in Python You can confirm our calculation using Python\u0026rsquo;s struct module, which lets you see the raw bytes of memory:\nimport struct # \u0026#39;f\u0026#39; indicates float (32 bits), pack converts to bytes # unpack converts bytes to an integer so we can see the hex bits = struct.unpack(\u0026#39;I\u0026#39;, struct.pack(\u0026#39;f\u0026#39;, -12.5))[0] print(hex(bits)) # Output: 0xc1480000 The math works!\nThe six steps we just went through fit into a single diagram, and it is worth keeping as a reference for when you want to redo the calculation with another number, since the sequence is always this one:\nThe six steps of converting −12.5, from the sign to the final bit pattern. Another Example: Decimal to IEEE 754 # The previous example was \u0026ldquo;clean\u0026rdquo; because the number had an exact binary representation. Now let us deal with a decimal number that cannot be represented exactly in binary, to see how the process handles fractions.\nGoal: Convert the number \\(0.085\\) to single precision format (32 bits).\nLet us follow a step-by-step that covers every detail:\nDetermine the Sign Since \\(0.085\\) is a positive number, the sign bit is 0.\nConvert to Binary Scientific Notation We need to write the number in the format \\(1.f \\times 2^e\\), where \\(1 \\le 1.f \u003c 2\\). We use successive multiplications by 2 (the same method as in the previous article) until the product exceeds 1, which identifies where the number normalizes:\n\\(0.085 \\times 2 = 0.17\\) → Bit 0 \\(0.17 \\times 2 = 0.34\\) → Bit 0 \\(0.34 \\times 2 = 0.68\\) → Bit 0 \\(0.68 \\times 2 = 1.36\\) → Bit 1 \\(\\leftarrow\\) First 1 found (0.36 left over) The first 1 appeared at the 4th multiplication, which means the point moves back 4 positions when normalizing. Therefore, the Real Exponent is \\(-4\\): \\[ 0.085 \\approx 1.36 \\times 2^{-4} \\] This line is a bridge, and it is worth reading carefully: it deliberately mixes a significand written in decimal (\\(1.36\\)) with an exponent in base 2. The calculation checks out (\\(1.36 / 16 = 0.085\\)), but the \\(1.36\\) is not yet what goes into memory: it itself needs to be converted to binary, and that is what the next step does. The stored significand will be derived from the bits that follow the 1., starting with the remaining fraction \\(0.36\\).\nCompute the Biased Exponent The standard uses a bias of 127. \\[ \\text{Stored Exponent} = -4 + 127 = 123 \\] Converting 123 to binary (8 bits): \\(123 = 01111011_2\\)\nConvert the Significand to Binary We need to convert the fraction \\(0.36\\) to binary by successively multiplying by 2 until we fill 23 bits (plus one extra bit for rounding):\n\\(0.36 \\times 2 = 0.72 \\to\\) Bit 0 \\(0.72 \\times 2 = 1.44 \\to\\) Bit 1 (0.44 left over) \\(0.44 \\times 2 = 0.88 \\to\\) Bit 0 \\(0.88 \\times 2 = 1.76 \\to\\) Bit 1 (0.76 left over) \\(0.76 \\times 2 = 1.52 \\to\\) Bit 1 (0.52 left over) \\(0.52 \\times 2 = 1.04 \\to\\) Bit 1 (0.04 left over) \\(0.04 \\times 2 = 0.08 \\to\\) Bit 0 \\(0.08 \\times 2 = 0.16 \\to\\) Bit 0 \\(0.16 \\times 2 = 0.32 \\to\\) Bit 0 \\(0.32 \\times 2 = 0.64 \\to\\) Bit 0 \\(0.64 \\times 2 = 1.28 \\to\\) Bit 1 (0.28 left over) \\(0.28 \\times 2 = 0.56 \\to\\) Bit 0 \\(0.56 \\times 2 = 1.12 \\to\\) Bit 1 (0.12 left over) \\(0.12 \\times 2 = 0.24 \\to\\) Bit 0 \\(0.24 \\times 2 = 0.48 \\to\\) Bit 0 \\(0.48 \\times 2 = 0.96 \\to\\) Bit 0 \\(0.96 \\times 2 = 1.92 \\to\\) Bit 1 (0.92 left over) \\(0.92 \\times 2 = 1.84 \\to\\) Bit 1 (0.84 left over) \\(0.84 \\times 2 = 1.68 \\to\\) Bit 1 (0.68 left over) \\(0.68 \\times 2 = 1.36 \\to\\) Bit 1 (0.36 left over, and the pattern restarts here) \\(0.36 \\times 2 = 0.72 \\to\\) Bit 0 (bit 21) \\(0.72 \\times 2 = 1.44 \\to\\) Bit 1 (bit 22) \\(0.44 \\times 2 = 0.88 \\to\\) Bit 0 (bit 23) \\(0.88 \\times 2 = 1.76 \\to\\) Bit 1 (bit 24, used for rounding) The fraction \\(0.36\\) is periodic in binary with period 20. The rounding bit 24 is 1 and the expansion continues after it, meaning that what was left over is strictly more than half a unit of the last stored bit. That is why bit 23 goes up from 0 to 1: it is rounding to nearest, plain and simple. (The even rule, Round to Nearest Even, only comes into play in an exact tie, when the remainder is exactly half; we will look at the rounding modes in detail later.) Final 23-bit significand: 01011100001010001111011\nFinal Result (Bits):\nPutting it all together: Sign 0, Exponent 01111011, Significand 01011100001010001111011. So: 0 01111011 01011100001010001111011\nExample: IEEE 754 to Decimal # Now that we know how to write to memory, let us do the reverse process: Reverse Engineering. How do we read an unknown bit pattern?\nGoal: Find out which decimal number is stored in the bit pattern: 11000000110110011001100110011010\nFollowing the reverse engineering of the previous section\u0026rsquo;s step-by-step:\nSlice the Bits We separate the binary into the three fields of the standard:\nSign (1 bit): 1 Exponent (8 bits): 10000001 Significand (23 bits): 10110011001100110011010 Analyze the Sign The bit is 1, so the number is Negative.\nDecode the Exponent The binary value 10000001 is equivalent to \\(128 + 1 = 129\\) in decimal. To find the real exponent, we subtract the bias (127): \\[ \\text{Real Exponent} = 129 - 127 = 2 \\] This means we will multiply the significand by \\(2^2\\).\nDecode the Significand The stored significand is 10110011.... We must remember that there is an implicit 1. in front and that each bit represents a negative power of 2 (\\(2^{-1}, 2^{-2}, 2^{-3}...\\)): \\[ \\begin{aligned} \\text{Value} \u0026= 1.(\\text{bits}) \\\\ \u0026= 1 + (1 \\times 2^{-1}) + (0 \\times 2^{-2}) + (1 \\times 2^{-3}) + (1 \\times 2^{-4}) + ... \\\\ \u0026= 1 + 0.5 + 0 + 0.125 + 0.0625 + ... \\\\ \u0026= 1.6875 + \\text{(remaining bits of the repeating pattern)} \\\\ \u0026\\approx 1.7 \\end{aligned} \\] (Note: The fraction 0.10110011... is a repeating expansion in binary, with the pattern 0110 repeating every 4 bits from the second bit on. Summing all the terms of the series, the fraction converges to exactly \\(0.7\\), confirming the significand \\(1.6875 + 0.0125 = 1.7\\).)\nFinal Calculation Now we apply the complete formula: \\[ \\begin{aligned} \\text{Number} \u0026\\approx (-1)^{\\text{sign}} \\times (1.\\text{significand}) \\times 2^{\\text{exponent}} \\\\ \u0026\\approx -1 \\times 1.7 \\times 2^2 \\\\ \u0026\\approx -1.7 \\times 4 \\\\ \u0026\\approx -6.8 \\end{aligned} \\] Therefore, those bits represent the approximate decimal value \\(\\mathbf{-6.8}\\). The value is approximate because of the nature of the binary representation of fractions.\nCheck your calculations We have already done the forward path (\\(-12.5\\) and \\(0.085\\)) and the backward path (the bits above). It is worth checking each of them in an interactive converter, which shows the three fields separately and the exact value that gets stored:\nFloat Exposed IEEE-754 Floating Point Converter Both accept either the decimal number or the bit pattern, so they work for checking both directions. Take the chance to play with the bits one by one and watch the value change: it is the best way to fix in your mind what each field does.\nRounding and \u0026ldquo;Catastrophic Cancellation\u0026rdquo; # So far, we have talked about how to store numbers. But the computer was made to calculate. And it is in addition and subtraction that the most subtle and dangerous errors occur.\nThe first point to understand is that the IEEE 754 standard does not simply \u0026ldquo;chop off\u0026rdquo; numbers that do not fit in the significand. It rounds. But it does so in a statistically clever way.\nRound to Nearest Even # In school, we learned a few variations of rounding (always up, always down, looking at the next digit), but for computers, these strategies can introduce a systematic bias.\nTo avoid this, the standard uses the Round to Nearest Even mode. The rule is simple: if the number is exactly in the middle (ending in \u0026hellip;5), we round to the nearest Even number.\n\\(1.5 \\to 2\\) (Rounded up) \\(2.5 \\to 2\\) (Rounded down) \\(3.5 \\to 4\\) (Rounded up) \\(4.5 \\to 4\\) (Rounded down) This way, over a long sequence of calculations, the upward and downward roundings tend to cancel out, keeping the statistical mean of the data much more faithful to reality.\nOther forms of rounding The IEEE 754 standard has other modes (such as always rounding up or down), used in Interval Arithmetic to guarantee that the real result lies within a known safety margin. But for general use, \u0026ldquo;Nearest Even\u0026rdquo; is the default.\nTo visualize the difference between the default and the alternatives, look at the table below applied to monetary values. Pay special attention to how each mode handles the \u0026ldquo;midpoint\u0026rdquo; (x.50) and negative numbers.\nRounding Mode $1.40 $1.60 $1.50 $2.50 $-1.50 Round-to-even $1 $2 $2 $2 $-2 Round-toward-zero $1 $1 $1 $2 $-1 Round-down $1 $1 $1 $2 $-2 Round-up $2 $2 $2 $3 $-1 What to notice in this table:\nThe .50 \u0026ldquo;Tie\u0026rdquo; (Round-to-even): Look at the $1.50 and $2.50 columns. Both were rounded to $2. This confirms the \u0026ldquo;Nearest Even\u0026rdquo; rule. If we used traditional school rounding, $1.50 would go to $2, but $2.50 would go to $3, creating an upward bias in the mean of the total data. Truncation (Round-toward-zero): This mode just chops off the decimal part. It is the default behavior when you convert int(1.9) in many languages: the result is 1, not 2. Direction on the Number Line (Down vs Up): Careful with negatives! Round-down always goes to the left on the number line (toward \\(-\\infty\\)). That is why \\(-1.50\\) goes down to \\(-2\\). It is the behavior of functions like math.floor() in Python. Round-up always goes to the right on the number line (toward \\(+\\infty\\)). That is why \\(-1.50\\) goes up to \\(-1\\). It is the behavior of functions like math.ceil() in Python. We can see these rules in action using a simple Python script. Note especially the difference between int and floor for negative numbers, and how round rounds to the nearest even.\nimport math values = [1.5, 2.5, -1.5] print(f\u0026#34;{\u0026#39;Num\u0026#39;:\u0026lt;6} {\u0026#39;int\u0026#39;:\u0026lt;6} {\u0026#39;floor\u0026#39;:\u0026lt;6} {\u0026#39;ceil\u0026#39;:\u0026lt;6} {\u0026#39;round\u0026#39;:\u0026lt;6}\u0026#34;) print(\u0026#34;-\u0026#34; * 40) for n in values: # int() = Truncation # floor() = Round down (Left) # ceil() = Round up (Right) # round() = Nearest Even print(f\u0026#34;{n:\u0026lt;6} {int(n):\u0026lt;6} {math.floor(n):\u0026lt;6} {math.ceil(n):\u0026lt;6} {round(n):\u0026lt;6}\u0026#34;) Output:\nNum int floor ceil round ---------------------------------------- 1.5 1 1 2 2 2.5 2 2 3 2 -1.5 -1 -2 -1 -2 The Danger: Catastrophic Cancellation # Rounding generates small errors in the last decimal place. Usually, this is acceptable. The real problem happens when these small errors are promoted to protagonists. This is called Catastrophic Cancellation.\nIt occurs when we subtract two very close numbers.\nImagine we have two values that should be exact, but suffered a tiny rounding error in the 7th decimal place:\n$$ \\begin{array}{r} x = 1.000000\\mathbf{5} \\\\ y = 1.000000\\mathbf{4} \\end{array} $$Doing \\(x - y\\): $$ 0.0000001 $$The computer now needs to Normalize this result (remember the \u0026ldquo;1 in front\u0026rdquo; rule?). It will move the point 7 places to the right to turn this into \\(1.0 \\times 10^{-7}\\).\nWhere is the problem? Originally, \\(x\\) and \\(y\\) had 7 digits of reliable precision. The result of the subtraction has only 1 digit of precision. The \u0026ldquo;good\u0026rdquo; bits (the \\(1.000...\\)) canceled each other out. What was left was only the \u0026ldquo;dirt\u0026rdquo; or \u0026ldquo;noise\u0026rdquo; that was at the end of the numbers. If those \\(...5\\) and \\(...4\\) were the fruit of earlier rounding (garbage), now your final result is 100% garbage, but the computer will treat it as a valid and precise number.\nQuadratic Formula Example Let us use the quadratic formula to illustrate this. The problematic case is when \\(b^2\\) is much larger than \\(4ac\\): then \\(\\sqrt{b^2 - 4ac}\\) gets very close to \\(\\lvert b \\rvert\\).\nNotice what happens with the root whose numerator is \\(-b + \\sqrt{b^2 - 4ac}\\): we are subtracting two nearly equal numbers, and the good digits cancel out (catastrophic cancellation). The other root, with \\(-b - \\sqrt{b^2-4ac}\\), has no problem at all, because there the values add up instead of canceling.\nSolution: rewrite the formula for the problematic root, rationalizing the numerator: $$x = \\frac{2c}{-b \\mp \\sqrt{b^2 - 4ac}}$$ It is the same root, algebraically, but now computed with a sum of values of the same sign in the denominator. It is a textbook example of how two formulas that are equivalent on paper can have completely different numerical behaviors.\nThis teaches us the golden lesson of Floating Point: Never blindly trust the subtraction of similar numbers.\nSpecial Values: Infinity, NaN and Signed Zero # So far, we have assumed that every bit pattern results in a valid number. But what happens if the exponent is filled entirely with ones (11111111) or entirely with zeros (00000000)?\nThe IEEE 754 standard reserves these extreme exponent values to represent special states. They are not normal numbers, but mathematical concepts and errors that need to be handled without crashing the computer.\nZero Has a Sign (+0 and -0) # In integers, zero is just zero. But in Floating Point, Zero is a very specific state defined by:\nExponent: All bits 0 (00000000). Significand: All bits 0 (empty). If the exponent is zero, but the significand has some bit 1, we are not looking at Zero, but at a Subnormal number, which we will see later.\nThis creates a seemingly bizarre situation: there is Positive Zero (0 00000000 000...) and Negative Zero (1 00000000 000...).\nFor most logical comparisons in your code, +0 == -0 returns True. The computer pretends they are equal.\nBut why does Negative Zero exist? Signed zero serves to preserve mathematical identities and solve complex problems (such as branch cuts in complex analysis). The simplest example is the function \\(1/x\\).\nIf \\(x\\) approaches zero from the positive side (\\(+0\\)), the result must be \\(+\\infty\\). If \\(x\\) approaches zero from the negative side (\\(-0\\)), the result must be \\(-\\infty\\). Without negative zero, we would lose the sign information in operations that result in \u0026ldquo;underflow\u0026rdquo; (numbers so small they become zero), breaking the mathematical identity \\(1/(1/x) = x\\).\nInfinity # What happens if you multiply \\(10^{30} \\times 10^{30}\\)? The result (\\(10^{60}\\)) is larger than the maximum value that fits in 32 bits. With integers, this would cause a silent Overflow and the number would \u0026ldquo;wrap around,\u0026rdquo; becoming negative or zero. A disaster for physical calculations.\nIn IEEE 754, if the number blows past the limit, it becomes Infinity.\nRepresentation: Exponent 11111111 (255) and Significand 0. Behavior: Infinity lets the mathematics continue. Depending on the configuration of the language or processor, \\(5 \\div 0\\) may raise an exception (error) or silently return \\(\\infty\\). In pure IEEE 754, the result is defined: \\(\\infty + 1000 = \\infty\\) \\(\\infty + \\infty = \\infty\\) \\(5 \\div 0 = \\infty\\) (Raises a \u0026ldquo;Division by Zero\u0026rdquo; flag, but the calculation goes on) A warning for those who will test this in the console: in plain Python, 5 / 0 raises ZeroDivisionError; the language chose to protect the programmer instead of following IEEE 754 to the letter. To see the behavior of the standard undisguised, use numpy:\nimport numpy as np np.float64(5) / np.float64(0) # RuntimeWarning: divide by zero encountered # np.float64(inf) Without this, it is easy to test in the console, see the exception and conclude that the standard says something else.\nThis design decision allows software to keep running even after an overflow, instead of aborting execution immediately.\nNaN (Not a Number) # What if we try to do something mathematically illegal?\nDivide \\(0 \\div 0\\). Compute \\(\\sqrt{-1}\\) (in the real domain). Subtract \\(\\infty - \\infty\\). The result is not a number. It is a NaN (Not a Number).\nRepresentation: Exponent 11111111 (255) and Significand different from 0. NaN propagates. Any calculation you do with a NaN results in NaN. \\[ 50 + \\text{NaN} = \\text{NaN} \\] How to detect a NaN NaN has a unique property: it is not equal to itself. Although you may see code using x != x to detect it, the robust way, recommended in production, is to use the standard library:\nimport math x = float(\u0026#39;nan\u0026#39;) # The right way: if math.isnan(x): print(\u0026#34;It\u0026#39;s a NaN!\u0026#34;) The purpose of NaN is to enable debugging. Instead of the program closing (crashing) in the middle of a long calculation, it propagates the NaN to the end, allowing you to trace the origin of the error.\nThe Limbo of Subnormals (Gradual Underflow) # Imagine you are dimming a light bulb (dimmer). You lower the light smoothly until\u0026hellip; click. It goes out completely.\nIn the early days of computing, numbers worked like this. The smallest possible \u0026ldquo;normal\u0026rdquo; number in 32 bits is \\(1.0 \\times 2^{-126}\\) (approximately \\(1.18 \\times 10^{-38}\\)). If you tried to divide this number by 2, the result would be smaller than the minimum representable. The computer rounded straight to Zero.\nThis was called Abrupt Underflow (Flush-to-Zero). The problem with it is that the subtraction \\(A - B\\) could result in Zero, even if \\(A\\) and \\(B\\) were different! This broke mathematical algorithms and caused instability in engineering software.\nThe Solution: Turning Off the Implicit Bit # To solve this, IEEE 754 created a \u0026ldquo;buffer zone\u0026rdquo; between the smallest normal number and absolute zero. These are the Subnormal Numbers (or Denormalized).\nThe rule is activated automatically when the Exponent is 00000000. When the processor sees the zeroed exponent, it changes its mode of operation:\nThe Hidden Bit becomes 0: Instead of assuming \\(1.\\text{significand}\\), the computer assumes \\(0.\\text{significand}\\). The Exponent is fixed at -126: Even though the bits are zero, the value of the exponent does not drop to -127 (which would be logical given the bias), but stays at -126 to maintain smooth continuity with normal numbers. The formula changes to: \\[ \\text{Value} = (-1)^{\\text{Sign}} \\times (0.\\text{Significand}) \\times 2^{-126} \\] The Practical Effect: Gradual Underflow # This lets numbers \u0026ldquo;disappear\u0026rdquo; gradually, bit by bit, instead of vanishing suddenly.\nSmallest Normal: \\(1.000...00 \\times 2^{-126}\\) Largest Subnormal: \\(0.111...11 \\times 2^{-126}\\) (Slightly smaller than the one above) \u0026hellip; Smallest Subnormal: \\(0.000...01 \\times 2^{-126}\\) (Only the last bit on) Zero: \\(0.000...00 \\times 2^{-126}\\) The difference this makes shows up immediately when the two scenarios are compared side by side. The figure again uses the reduced format, so that the marks fit on the page, but the structure is exactly that of IEEE 754:\nWithout subnormals, there is nothing between zero and the smallest normal, and the gap is larger than the next step. With subnormals, that gap is filled at the same step. Note the detail the drawing makes evident: without subnormals, the biggest hole in the line would be right around zero, which is where you can least afford a hole, because that is where subtractions of close numbers end up.\nThe Cost of Subnormality # Nothing comes for free. By using subnormal numbers, we pay a price: Loss of Precision.\nRemember that scientific notation serves to keep precision always high. But with subnormals, we start filling the beginning of the number with zeros (\\(0.00001...\\)). These zeros take up space in the Significand and reduce the number of \u0026ldquo;useful\u0026rdquo; (significant) bits. The smaller the subnormal number, the less reliable it is. But, according to the standard, it is better to have a number with little precision than a false zero.\nHardware Curiosity Computing with subnormals is hard. On many older processors (and some modern ones), operations with subnormals are handled in software (microcode) rather than directly in hardware, which can be dozens of times slower. That is why some high-performance compilers have flags such as -ffast-math that enable \u0026ldquo;Flush-to-Zero,\u0026rdquo; sacrificing mathematical precision in exchange for raw speed in games or simulations.\nThe Precision Error (0.1 + 0.2 != 0.3) # If you open the Python console (or JavaScript, C, Java\u0026hellip;) and type:\nprint(0.1 + 0.2 == 0.3) # False The answer will be False. And if you ask to see the result of the sum:\nprint(0.1 + 0.2) # 0.30000000000000004 Where did that ...4 at the end come from? Did the computer get an elementary-school sum wrong? No. It did the calculation perfectly, within the limits of the binary system.\nThe Problem of Repeating Decimals # To understand this, we need to go back to school. In decimal (base 10), we know that some fractions cannot be represented in a finite way.\nWe have a repeating decimal. If you add \\(0.333 + 0.333 + 0.333\\), the result is \\(0.999\\), not \\(1.0\\). There was a loss of precision because we cut off the repeating decimal.\nThe same happens in binary, but with different numbers. In binary, we can only represent finitely the fractions that are sums of powers of 2 (like \\(0.5, 0.25, 0.125\\)).\nThe case of 0.1:\nLet us try to convert decimal \\(0.1\\) to binary:\n\\(0.1 \\times 2 = 0.2\\) \\(\\to\\) Digit 0 \\(0.2 \\times 2 = 0.4\\) \\(\\to\\) Digit 0 \\(0.4 \\times 2 = 0.8\\) \\(\\to\\) Digit 0 \\(0.8 \\times 2 = \\mathbf{1}.6\\) \\(\\to\\) Digit 1 (0.6 left over) \\(0.6 \\times 2 = \\mathbf{1}.2\\) \\(\\to\\) Digit 1 (0.2 left over) Oops! We are back to 0.2 (Step 2). The cycle repeats forever: \\(0.00011001100110011...\\). The number \\(0.1\\) is a repeating expansion in binary.\nAnd how does this look in the 32 bits, in practice? It is worth taking the whole path, the same one we took with \\(0.085\\) and with \\(-12.5\\), now with the most famous number on the subject:\nThe first 1 appeared at the 4th multiplication, so the real exponent is \\(-4\\) and the fraction \\(0.6\\) is left for the significand. Stored exponent: \\(-4 + 127 = 123 = 01111011_2\\). Significand: \\(0.6\\) in binary is 1001 repeated forever. Cutting at 23 bits would give 10011001100110011001100, but the next bit is 1 (and the expansion continues), so the last bit goes up: 10011001100110011001101. Assembly: 0 01111011 10011001100110011001101. And here the two ends of the series meet. That bit pattern that the article on real numbers extracted from the hardware with struct, without explaining where it came from, was exactly this one:\nimport struct bits = struct.unpack(\u0026#39;I\u0026#39;, struct.pack(\u0026#39;f\u0026#39;, 0.1))[0] print(f\u0026#34;{bits:032b}\u0026#34;) # 00111101110011001100110011001101 print(hex(bits)) # 0x3dcccccd Split the string into 0, 01111011 and 10011001100110011001101 and compare it with the assembly above. Bit by bit, it is the calculation we just did on paper.\nThe 23-Bit Guillotine # As we saw in the previous sections, the IEEE 754 standard (Single Precision) has only 23 bits in the significand (plus the hidden bit). It does not have infinite room to store this repeating expansion. At some point, the computer has to bring out the \u0026ldquo;scissors\u0026rdquo; and stop writing.\nWhen it does this, it is not storing \\(0.1\\). It is storing a number extremely close, but slightly smaller or larger (depending on the rounding).\nIn 32 bits, which is the format dissected in this article:\n0.1 becomes \\(0.100000001490116\\ldots\\) 0.2 becomes \\(0.200000002980232\\ldots\\) Now, pay attention to a precision switch that often goes unnoticed: the famous output, 0.30000000000000004, which you see in the Python console, is 64 bits, the format Python uses for every float. There, the approximations are much finer:\n0.1 becomes \\(0.1000000000000000055511\\ldots\\) 0.2 becomes \\(0.2000000000000000111022\\ldots\\) The number of digits changes, not the phenomenon. In either format, the errors of the two approximations add up and the result escapes the margin the computer uses to decide how to display the number on screen, revealing the \u0026ldquo;garbage\u0026rdquo; at the end: \\(0.30000000000000004\\) in 64 bits.\nThe error that accumulates Everything we have seen so far is the error of a single operation. What usually brings down real code is accumulation: the error of each step added to that of the next step, thousands of times.\ntotal = 0.0 for _ in range(10): total += 0.1 print(total) # 0.9999999999999999 print(total == 1.0) # False Adding \\(0.1\\) ten times does not give \\(1.0\\). Each term carries its little error, and ten of them added together already show up in the last digit. Now imagine a simulation loop with a million iterations: that is why, when you can choose, you prefer total = n * step to adding step repeatedly.\nHow Wrong Can It Be? Machine Epsilon # So far we have talked about the error qualitatively: \u0026ldquo;very close,\u0026rdquo; \u0026ldquo;almost equal,\u0026rdquo; \u0026ldquo;a bit of garbage at the end.\u0026rdquo; There is a number that answers that question exactly, and it has a name: the machine epsilon.\nThe definition is straightforward: it is the distance between \\(1.0\\) and the next representable number. In 32 bits, the significand stores 23 bits, so this step is \\(2^{-23} \\approx 1.19 \\times 10^{-7}\\). In 64 bits, with 52 bits of significand, it is \\(2^{-52} \\approx 2.22 \\times 10^{-16}\\).\nPython hands you this value ready-made, along with the rest of the spec sheet of your machine\u0026rsquo;s float:\nimport sys print(sys.float_info.epsilon) # 2.220446049250313e-16 print(sys.float_info.max) # 1.7976931348623157e+308 print(sys.float_info.min) # 2.2250738585072014e-308 print(sys.float_info.dig) # 15 print(sys.float_info.mant_dig) # 53 These are exactly the numbers from the 32 × 64 bits table we saw at the beginning, now read from your processor instead of memorized. Two deserve attention. mant_dig is 53, not 52, because the hidden bit counts as precision. And min is the smallest normal number: below it there is still the subnormal zone we just walked through.\nThe epsilon becomes visible the hard way:\nimport sys print(1.0 + sys.float_info.epsilon) # 1.0000000000000002 print(1.0 + sys.float_info.epsilon / 2) # 1.0 Half of the epsilon simply vanishes. There is nowhere to store the result, and it rounds back to \\(1.0\\).\nEpsilon and ULP are not the same thing The epsilon is the step right after \\(1.0\\). The step around an arbitrary number has another name: ULP (Unit in the Last Place). And it grows along with the number. It is the relative precision we discussed in the anatomy of the format, now with concrete values:\nimport math print(math.ulp(1.0)) # 2.220446049250313e-16 print(math.ulp(1e6)) # 1.1641532182693481e-10 print(math.ulp(1e16)) # 2.0 Around \\(10^{16}\\), the distance between two neighboring floats is already 2. The consequence is direct: 1e16 + 1 == 1e16 returns True. That 1 has nowhere to be stored.\nWith this, the question \u0026ldquo;how wrong can my float be?\u0026rdquo; finally has a numerical answer: in a well-executed operation, the relative error is at most half the epsilon. It seems negligible, and it is, until you chain millions of these operations, as in the summation loop we just saw.\nAnd it is precisely because the error exists, however tiny, that comparing floats requires care.\nPractical Solution: Never use == with Floats Squeezing infinitely many real numbers into a finite number of bits requires approximation. Therefore, strict equality (==) almost never works for computed results. The Right Way: Check whether the numbers are \u0026ldquo;close enough\u0026rdquo; to each other, using a tiny margin of error (called tolerance).\nimport math # Correct way to compare a = 0.1 + 0.2 b = 0.3 if math.isclose(a, b): print(\u0026#34;They are equal (for practical purposes)\u0026#34;)With the isclose function, you can adjust the tolerance according to the precision needed for your specific domain. Here is an example in Python:\nimport math a = 0.1 + 0.2 b = 0.3 # Check whether \u0026#39;a\u0026#39; and \u0026#39;b\u0026#39; are close enough if math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0): print(\u0026#34;They are equal (for practical purposes)\u0026#34;) In this example, rel_tol=1e-9 defines the relative tolerance, allowing a and b to be considered equal if they are within a difference of \\(10^{-9}\\) times the value of b. The parameter abs_tol=0.0 indicates that there is no additional absolute tolerance. Adjust these values according to the precision needed for your use case.\nFor Money:\nIf you are dealing with money, cents matter and rounding errors are unacceptable. Never use Float for money. Use Decimal types (in Python) or BigDecimal (in Java), which work in base 10 and do not suffer from this problem.\nSummary Table # Everything this article established about 32-bit IEEE 754, on a single page:\nConcept Rule Example from the article Fixed-point limit a locked point sacrifices range or precision 16 + 16 bits: ceiling of \\(2^{16}-1\\), step of \\(2^{-16}\\) Floating point scientific notation in binary: \\(\\pm 1.m \\times 2^{e}\\) \\(1100.1_2 = 1.1001_2 \\times 2^3\\) Fields (32 bits) 1 sign + 8 exponent + 23 significand 1 10000010 10010000000000000000000 Reading formula \\((-1)^{S} \\times 1.M \\times 2^{E-127}\\) \\(-1 \\times 1.7 \\times 2^2 \\approx -6.8\\) Sign bit sign-magnitude, not two\u0026rsquo;s complement flipping only this bit swaps \\(+x\\) for \\(-x\\) Hidden bit the 1. of normalization takes no space 23 bits stored, 24 of precision Exponent bias stored \\(=\\) real \\(+\\) 127 real exponent \\(3 \\to 130 = 10000010_2\\) Decimal \\(\\to\\) IEEE 754 sign, binary, normalize, bias, significand, assemble \\(-12.5 \\to\\) 0xC1480000 IEEE 754 \\(\\to\\) decimal split the three fields and apply the formula 1 10000001 10110011... \\(\\to -6.8\\) Default rounding to nearest; only in an exact tie does it go to even \\(0.085\\): bit 23 goes up from 0 to 1 Catastrophic cancellation subtracting nearly equal values destroys the good digits quadratic formula when \\(b^2 \\gg 4ac\\) Exponent all 1 special value: \\(\\pm\\infty\\) (significand zero) or NaN \\(0/0\\) returns NaN NaN is not equal to anything, not even itself nan == nan is False; use math.isnan Exponent all 0 subnormal zone: hidden bit becomes 0, exponent locks at \\(-126\\) \\(0.M \\times 2^{-126}\\) Why subnormals avoid abrupt underflow and guarantee \\(A - B = 0 \\iff A = B\\) the \u0026ldquo;dimmer\u0026rdquo; that fades out gradually Signed zero \\(+0\\) and \\(-0\\) coexist, and are equal in comparison 0.0 == -0.0 is True Repeating expansion in binary a decimal fraction that is not a sum of powers of 2 does not fit \\(0.1 \\to 0.0\\overline{0011}_2\\) Machine epsilon step right above \\(1.0\\): \\(2^{-23}\\) in 32 bits, \\(2^{-52}\\) in 64 1.0 + eps/2 returns 1.0 ULP step around any number; grows with it math.ulp(1e16) is \\(2.0\\) Error accumulation adding \\(n\\) terms adds \\(n\\) little errors \\(0.1\\) added ten times gives 0.9999999999999999 Comparing floats tolerance, never == math.isclose(0.1 + 0.2, 0.3) Money Decimal, never float cents demand exact base 10 Conclusion # Throughout this article, we went from the simple arithmetic of counting oranges with integers to the internal structure of the IEEE 754 standard.\nNow you understand that when the computer displays \\(0.30000000000000004\\), it is not \u0026ldquo;wrong\u0026rdquo; or \u0026ldquo;broken.\u0026rdquo; It is just being finite.\nFloating Point is, above all, an exercise in the engineering of trade-offs:\nThe Range Trade-off: To represent the mass of the Sun and the mass of a proton in the same 32-bit variable, we had to accept that the point floats. The Precision Trade-off: To gain that astronomical range, we had to accept that we cannot store all real numbers. There are holes in the computer\u0026rsquo;s number line, and numbers like \\(0.1\\) fall into those holes, needing to be rounded. The Speed Trade-off: The Exponent Bias allows comparing two positive floats with the same simple, fast circuits that compare unsigned integers, and the Hidden Bit gives one extra bit of precision without taking up storage space. What to take into your professional life? # Now that you know the \u0026ldquo;Matrix\u0026rdquo; of real numbers, here are the golden rules to survive:\nKnow where to use Float: It is perfect for 3D graphics, scientific calculations, game physics and statistics, where speed and range outweigh the need for absolute precision at the 15th decimal place. Know where NOT to use float: Be careful using float or double for banking or accounting systems. Money demands exact decimal precision. Use Decimal libraries. Beware of Equality: Never compare floats with ==. Always use a margin of error (tolerance). Beware of Subtraction: Remember \u0026ldquo;Catastrophic Cancellation\u0026rdquo; when subtracting very close numbers. The computer is a precise machine, but within physical limits that the programmer needs to know.\nThat is it for this article, folks. See you next time!\n","date":"2026-04-03","externalUrl":null,"permalink":"/2026/04/ieee-754-real-number-representation/","section":"Blog","summary":"How can the same 32 bits hold the distance between galaxies and the diameter of an atom? We dissect IEEE 754: sign, biased exponent and significand with a hidden bit, hand conversions, infinity, NaN, subnormals and machine epsilon.","title":"Inside IEEE 754: How the Computer Stores Real Numbers (and Why It Gets Them Wrong)","type":"posts"},{"content":"","date":"2026-04-03","externalUrl":null,"permalink":"/categories/number-systems/","section":"Categories","summary":"","title":"Number Systems","type":"categories"},{"content":"","date":"2026-04-03","externalUrl":null,"permalink":"/categories/sistemas-de-numeracao/","section":"Categorias","summary":"","title":"Sistemas de Numeração","type":"categories"},{"content":"","date":"2026-04-01","externalUrl":null,"permalink":"/tags/float/","section":"tags","summary":"","title":"float","type":"tags"},{"content":"","date":"2026-04-01","externalUrl":null,"permalink":"/tags/hexadecimal/","section":"tags","summary":"","title":"hexadecimal","type":"tags"},{"content":"","date":"2026-04-01","externalUrl":null,"permalink":"/tags/precisao/","section":"tags","summary":"","title":"precisão","type":"tags"},{"content":"","date":"2026-04-01","externalUrl":null,"permalink":"/tags/precision/","section":"tags","summary":"","title":"precision","type":"tags"},{"content":" Throughout this series, we have discussed how to convert, operate on and manipulate integers in different bases. We saw that, for integers, conversion is always exact: an integer in one base always has an exact counterpart in another.\nHowever, the real world is not made only of integers. Measurements, probabilities and physical constants depend on fractional (real) numbers. The question is: how do we represent \\(0.1\\) or \\(3.14159\\) using only zeros and ones?\nIn this article, we will extend our methods to handle the fractional part. You will see that, although the logic is similar, fractions bring a new challenge: precision. Unlike integers, not every finite decimal fraction has a finite representation in binary, which gives rise to the famous rounding errors in computing.\nThe general rule remains the same: we use the decimal base as a bridge.\nDecimal \\(\\to\\) Base \\(r\\): we use Successive Multiplications. Base \\(s\\) \\(\\to\\) Decimal: we use Positional Expansion (with negative exponents). From Decimal to Any Base (Successive Multiplications) # To convert the fractional part from decimal to binary, octal or hexadecimal, we use the method of successive multiplications. Unlike the integer part (where we divided), here we multiply by the target base to \u0026ldquo;push\u0026rdquo; the digits into the integer part.\nThe Algorithm:\n1 Multiply by the base Multiply the fractional part of the number by the target base \\(r\\). 2 Keep the integer part The integer part of the result becomes the next digit in the new base. 3 Repeat with the fraction Take only the remaining fractional part and repeat the process. 4 Stop Stop when the result is zero (exact) or when you reach the desired precision (in the case of repeating fractions). Example: Convert \\(0.625_{10}\\) to binary (\\(2\\)) with 3 fractional digits \\(0.625 \\times 2 = \\mathbf{1}.25\\) → we keep 1, \\(0.25\\) remains \\(0.25 \\times 2 = \\mathbf{0}.50\\) → we keep 0, \\(0.50\\) remains \\(0.50 \\times 2 = \\mathbf{1}.00\\) → we keep 1, \\(0.00\\) remains Since the fraction reached zero, the conversion is exact. Reading from top to bottom: $$ 0.625_{10} = 0.101_2 $$The big challenge here is fractions that look simple in decimal but are infinite (repeating) in binary. Take the case of \\(0.3\\):\nExample: Convert \\(0.3_{10}\\) to binary (\\(2\\)) with 5 fractional digits \\(0.3 \\times 2 = \\mathbf{0}.6\\) → we keep 0 \\(0.6 \\times 2 = \\mathbf{1}.2\\) → we keep 1 \\(0.2 \\times 2 = \\mathbf{0}.4\\) → we keep 0 \\(0.4 \\times 2 = \\mathbf{0}.8\\) → we keep 0 \\(0.8 \\times 2 = \\mathbf{1}.6\\) → we keep 1 Note that \\(0.6\\) remains, which would take us back to step 2. The result is a repeating fraction (\\(0.010011001...\\)). To get the 5 digits requested in the statement of the example, we truncate: $$ 0.3_{10} \\approx 0.01001_2 $$This \u0026ldquo;going back to step 2\u0026rdquo; is what produces the repeating fraction, and it is clearer when drawn. In the figure, the five steps appear in sequence, each with the bit it produces and the remainder it leaves. The red arrow marks the moment when the remainder repeats and the process enters a loop:\nThe five steps of the conversion of 0.3: at the fifth, the remainder is 0.6 again and the cycle starts over. It is this return that makes the expansion infinite. Notice that the cycle has four steps, not five: step 1 happens only once, and what repeats is the block formed by steps 2 to 5. That is why the result is written as \\(0.0\\) followed by 1001 repeated forever.\nWhat If We Have Both an Integer and a Fractional Part? # The strategy is the classic \u0026ldquo;divide and conquer\u0026rdquo;. You don\u0026rsquo;t need a new method; just split the number into two smaller problems:\n1 Convert the integer part Convert the integer part using Successive Divisions (as seen in the article on integer conversion). 2 Convert the fractional part Convert the fractional part using Successive Multiplications. 3 Join the two Join the two results with the separator point. Mixed Example: Convert \\(6.512_{10}\\) to binary (\\(2\\)) with 5 digits of precision Part 1: The Integer (\\(6\\))\nWe use divisions: $$ 6 \\div 2 = 3 \\quad (\\text{remainder } \\mathbf{0}) \\\\ 3 \\div 2 = 1 \\quad (\\text{remainder } \\mathbf{1}) \\\\ 1 \\div 2 = 0 \\quad (\\text{remainder } \\mathbf{1}) $$ Integer Result: \\(110_2\\)\nPart 2: The Fraction (\\(0.512\\))\nWe use multiplications:\n\\(0.512 \\times 2 = \\mathbf{1}.024\\) → we keep 1 \\(0.024 \\times 2 = \\mathbf{0}.048\\) → we keep 0 \\(0.048 \\times 2 = \\mathbf{0}.096\\) → we keep 0 \\(0.096 \\times 2 = \\mathbf{0}.192\\) → we keep 0 \\(0.192 \\times 2 = \\mathbf{0}.384\\) → we keep 0 Fractional Result: \\(0.10000..._2\\)\nPart 3: Union $$ 6.512_{10} \\approx 110.10000_2 $$Notice the \u0026ldquo;approximately\u0026rdquo; sign. Stop and walk the path back: \\(110.10000_2\\) is worth exactly \\(4 + 2 + 0.5 = 6.5\\). That is, stopping at 5 binary digits lost \\(0.012\\) along the way. There was no arithmetic mistake: the number of digits ran out before the fraction did. This is the first time in this article that you see truncation error happening, and it is exactly this mechanism that lies behind all the floating-point oddities at the end of the text.\nVisualizing in Python (The X-Ray of Memory) # Let\u0026rsquo;s use Python to see how real numbers are stored in the computer\u0026rsquo;s memory. Remember that, internally, real numbers are represented in floating point, following the IEEE 754 standard.\nThe IEEE 754 standard will be detailed in future articles. But the basics you need to know now is that a floating-point real number is divided into three main fields:\nSign (1 bit): Indicates whether the number is positive (0) or negative (1). Exponent (8 bits for a 32-bit float): Stores the \u0026ldquo;adjusted\u0026rdquo; exponent (with a bias). The bias for 32-bit floats is 127. Significand (23 bits for a 32-bit float): Stores the fractional part of the number, with an implicit \u0026ldquo;hidden bit\u0026rdquo;. Here we are using 32 bits just as an example. For other floating-point sizes (such as the 64-bit double), the field sizes are different, but the concept is the same. Again, this will be explored in depth in future articles. For now, let\u0026rsquo;s focus on how to visualize these bits in Python.\nPerhaps your first reaction is to try the built-in bin() function, which we used exhaustively in the articles on integers. But if you try to run bin(0.625), Python will give you an error:\nTypeError: 'float' object cannot be interpreted as an integer\nThis happens because the bin() function was made to show the exact representation of integers. To see how a real number (float) is actually stored in the 32 bits of memory, looking at the Sign, Exponent and Significand fields, we need a different approach.\nWe will use the struct library to take the float value, pack it into raw bytes and then read those same bytes as if they were an integer. This gives us an \u0026ldquo;X-Ray\u0026rdquo; of the hardware. The struct library is used to convert between Python and C representations, allowing you to manipulate binary data directly.\nAttention: 32-bit visualization for didactic purposes The following code uses the 'f' format of struct, which packs the number into 32 bits (the C float). Remember that Python stores its floats with 64 bits ('d'), using 11 bits of exponent and 52 of significand. The structure of sign, exponent and significand is identical; only the field sizes differ.\nAbout byte order The 'f' and 'I' formats without a prefix use the machine\u0026rsquo;s native byte order. Since both ends use the same order, the resulting bit pattern is the same on any platform; the code below is portable as it stands.\nStill, in an article about binary representation it is worth knowing the explicit form: struct.unpack('\u0026lt;I', struct.pack('\u0026lt;f', number)) fixes little-endian (and '!' would fix network order). The advantage is documenting the intent and protecting against someone who, one day, changes only one of the two format strings. Mixing the orders ('\u0026lt;I' with '\u0026gt;f') is the only way to get wrong bits here.\nimport struct def view_float_bits(number): # \u0026#39;f\u0026#39; = float (32 bits, IEEE 754 standard) # \u0026#39;I\u0026#39; = unsigned int (32-bit unsigned integer) # pack: converts the float number to raw bytes # unpack: reads those raw bytes as if they were an integer int_bits = struct.unpack(\u0026#39;I\u0026#39;, struct.pack(\u0026#39;f\u0026#39;, number))[0] # Formats as a string of 32 zeros and ones bit_string = f\u0026#34;{int_bits:032b}\u0026#34; # Let\u0026#39;s print it in a didactic way, separating the fields sign = bit_string[0] exponent = bit_string[1:9] significand = bit_string[9:] print(f\u0026#34;\\nAnalyzing the number: {number}\u0026#34;) print(f\u0026#34;Raw Bits: {bit_string}\u0026#34;) print(f\u0026#34;Split: [S:{sign}] [Exp:{exponent}] [Sig:{significand}]\u0026#34;) # Test 1: Purely fractional number (previous example) view_float_bits(0.625) # Test 2: Number with integer and fractional parts view_float_bits(12.5) # Test 3: Repeating number (0.1) view_float_bits(0.1) # Test 4: Negative number (-3.75) view_float_bits(-3.75) Output:\nAnalyzing the number: 0.625 Raw Bits: 00111111001000000000000000000000 Split: [S:0] [Exp:01111110] [Sig:01000000000000000000000] Analyzing the number: 12.5 Raw Bits: 01000001010010000000000000000000 Split: [S:0] [Exp:10000010] [Sig:10010000000000000000000] Analyzing the number: 0.1 Raw Bits: 00111101110011001100110011001101 Split: [S:0] [Exp:01111011] [Sig:10011001100110011001101] Analyzing the number: -3.75 Raw Bits: 11000000011100000000000000000000 Split: [S:1] [Exp:10000000] [Sig:11100000000000000000000] What are we seeing?\nTo follow the calculations below, consider that multiplying by \\(2^e\\) is the same as moving the point \\(e\\) places to the right (if \\(e\\) is positive) or to the left (if \\(e\\) is negative).\nIn the case of 0.625: The exponent is 01111110 (126 decimal). Since the bias is 127, the actual exponent is \\(126 - 127 = -1\\). The stored significand is 01000.... Remember the \u0026ldquo;Hidden Bit\u0026rdquo;: the computer assumes a \u0026ldquo;1.\u0026rdquo; in front. So the actual significand is \\(1.01\\). Final calculation: \\(1.01_2 \\times 2^{-1} = 0.101_2 = 0.625\\). It matches! In the case of 12.5: The exponent is 10000010 (130 decimal). Actual exponent: \\(130 - 127 = 3\\). The stored significand is 1001000.... Putting the implicit \u0026ldquo;1.\u0026rdquo; in front, we have \\(1.1001\\). Final calculation: \\(1.1001_2 \\times 2^3\\). Moving the point 3 places to the right gives \\(1100.1_2\\), which is exactly \\(12.5\\) decimal. In the case of 0.1 (The Revealing One): The exponent is 01111011 (123 decimal). Actual exponent: \\(123 - 127 = -4\\). Look at the significand: 100110011001.... Do you notice the repeating pattern? Here is the visual proof that 0.1 is a repeating fraction in binary. Note the last bit 1. The computer had to round the value because the sequence was infinite. It is this small \u0026ldquo;dirt\u0026rdquo; at the end that causes errors such as \\(0.1 + 0.2 \\neq 0.3\\). Final calculation: \\(1.100110011001..._2 \\times 2^{-4}\\). Moving the point 4 places to the left gives \\(0.0001100110011..._2\\), which is the approximate binary representation of \\(0.1\\). In the case of -3.75: Sign: 1 (Negative). The exponent is 10000000 (128 decimal). Actual exponent: \\(128 - 127 = 1\\). This means multiplying by \\(2^1\\). The stored significand is 111000.... Adding the implicit \u0026ldquo;1.\u0026rdquo;, we have \\(1.111\\). Final calculation: \\(-1 \\times 1.111_2 \\times 2^1\\). Moving the point 1 place to the right gives \\(-11.11_2\\). Translating: \\(11_2\\) is 3 and \\(0.11_2\\) is \\(0.5 + 0.25 = 0.75\\). Result: \\(-3.75\\). If you had trouble, don\u0026rsquo;t worry. The floating-point representation has many details. That is why we dedicate an article in this series to the IEEE 754 standard, where we will detail each field (sign, exponent and significand) and explain the special cases such as NaN, Infinity and Subnormal numbers.\nFrom Any Base to Decimal (Positional Expansion) # To find the decimal value of a fraction in another base, we use positional expansion. The difference is that, after the point, the positions have negative exponents: \\(b^{-1}, b^{-2}, b^{-3}\\), and so on.\n$$ 0.d_1 d_2 d_3 \\dots _b = d_1 \\cdot b^{-1} + d_2 \\cdot b^{-2} + d_3 \\cdot b^{-3} + \\dots $$ Example: Convert \\(0.101_2\\) to decimal (\\(10\\)) $$ \\begin{align*} 0.101_2 \u0026 = 1 \\times 2^{-1} + 0 \\times 2^{-2} + 1 \\times 2^{-3} \\\\ \u0026 = 1 \\times 0.5 + 0 \\times 0.25 + 1 \\times 0.125 \\\\ \u0026 = 0.5 + 0 + 0.125 \\\\ \u0026 = 0.625_{10} \\end{align*} $$ The conversion is exact, since the sum is finite.\nThe General Case: Using the Decimal Bridge # If we need to convert between any two bases (e.g., Base 6 to Octal), the safest path is:\nConvert the source fraction to Decimal (Expansion). Convert from Decimal to the target (Multiplication). Example: Convert \\(0.32_4\\) to binary (\\(2\\)) with 3 fractional digits Convert Base 4 to Decimal: $$ \\begin{align*} 0.32_4 \u0026 = 3 \\times 4^{-1} + 2 \\times 4^{-2} \\\\ \u0026 = 3 \\times 0.25 + 2 \\times 0.0625 = 0.75 + 0.125 = 0.875_{10} \\end{align*} $$ Convert Decimal (\\(0.875\\)) to Binary:\n\\(0.875 \\times 2 = \\mathbf{1}.75\\) → digit 1 \\(0.75 \\times 2 = \\mathbf{1}.50\\) → digit 1 \\(0.50 \\times 2 = \\mathbf{1}.00\\) → digit 1 See that the fraction reached zero, so the conversion is exact.\nFinal result: $$ 0.32_4 = 0.111_2 $$ Example: Convert \\(0.52_6\\) to octal (\\(8\\)) with 3 fractional digits Convert Base 6 to Decimal: $$ \\begin{align*} 0.52_6 \u0026 = 5 \\times 6^{-1} + 2 \\times 6^{-2} \\\\ \u0026 = 5 \\times 0.1666... + 2 \\times 0.0277... \\approx 0.8888_{10} \\end{align*} $$ (Note: For maximum precision, keep the fractions: \\(5/6 + 2/36 = 32/36 = 8/9\\))\nConvert Decimal (\\(8/9 \\approx 0.888...\\)) to Octal:\n\\(0.8888... \\times 8 = 7.1111...\\) → digit 7 \\(0.1111... \\times 8 = 0.8888...\\) → digit 0 \\(0.8888... \\times 8 = 7.1111...\\) → digit 7 Observe that the pattern repeats, indicating a repeating fraction in octal.\nApproximate result: $$ 0.52_6 \\approx 0.707_8 $$ Bases Related by Powers (Shortcuts) # As with integers, if the bases are powers of one another (such as 2, 8 and 16), we can convert by grouping or expanding bits.\nGrouping starts at the point For mixed numbers (which have an integer and a fractional part), the golden rule is: always start at the point and move toward the ends.\nImagine the point as a mirror. The grouping must \u0026ldquo;move away\u0026rdquo; from it in both directions:\nInteger part (left of the point): group from right to left \\(\\leftarrow\\). If the last group is incomplete, pad with zeros on the left. Fractional part (right of the point): group from left to right \\(\\rightarrow\\). If the last group is incomplete, pad with zeros on the right. Example: Convert \\(0.A7_{16}\\) to binary (\\(2\\)) We expand each hexadecimal digit of the fractional part into 4 bits: $$ A_{16} = 1010_2, \\quad 7_{16} = 0111_2 $$Result: $$ 0.A7_{16} = 0.10100111_2 $$Now, here is an example where zero padding is necessary.\nExample: Convert \\(11011.11_2\\) to Hexadecimal (\\(16\\)) The starting point is the point. We will group 4 bits in each direction. $$ \\leftarrow \\mathbf{11011} \\quad . \\quad \\mathbf{11} \\rightarrow $$Left Side (Integer):\nGroup 1 (near the point): \\(1011\\) \\(\\to B\\) Group 2 (left end): \\(1\\) is left over. We pad with zeros on the left: \\(0001\\) \\(\\to 1\\) Right Side (Fraction):\nGroup 1 (near the point): We only have \\(11\\). Since we need 4 bits, we pad with zeros on the right: \\(1100\\) \\(\\to C\\) Final Result: $$ 1B.C_{16} $$One more example to reinforce the concept:\nNew Example: Convert \\(0.11011_2\\) to Hexadecimal (\\(16\\)) We need to group the bits in blocks of 4, starting at the point and moving right.\nOriginal number: $$ 0.11011_2 $$Group 1: The first 4 bits: $$ 1101_2 = D_{16} $$Group 2: Only the bit 1 is left. Since we need a block of 4, we pad with zeros on the right: $$ 1 \\rightarrow 1000_2 = 8_{16} $$(Note: If we padded on the left, we would get \\(0001=1\\), which would be wrong, since the original bit has weight \\(2^{-5}\\), not \\(2^{-8}\\))\nFinal result: $$ 0.11011_2 = 0.D8_{16} $$ Floating Point in Python: Curiosities and Limitations # Besides what we saw earlier in the section \u0026ldquo;Visualizing in Python (The X-Ray of Memory)\u0026rdquo;, there are some other interesting curiosities about how Python handles real numbers and their bases.\nUnlike integers, where Python gives us easy prefixes such as 0b and 0x, dealing with non-decimal bases for fractional numbers requires different tools.\nThere Is No \u0026ldquo;Fractional Binary Literal\u0026rdquo; # You cannot write something like x = 0b1.101 in Python. The language does not support that syntax natively.\nIf you need to convert a fractional binary string (such as \u0026ldquo;10.11\u0026rdquo;) to float, you will have to do it manually or use libraries. The computer\u0026rsquo;s internal logic is binary, but Python abstracts that away from the programmer.\nHexadecimal Floats # Python lets you convert floats to hexadecimal and vice versa natively. This is useful for programmers who need to know exactly what is stored in memory, without the rounding of the decimal display.\nThe format uses the notation 0x...p..., where p indicates a power of 2 (similar to scientific notation, but binary).\n# Converting a float to its exact representation in Hex x = 10.5 print(x.hex()) # Output: \u0026#39;0x1.5000000000000p+3\u0026#39; # Means: 1.5 (hex) * 2^3 # Converting back (Hex to Float) y = float.fromhex(\u0026#39;0x0.3p10\u0026#39;) # 0.1875 * 1024 print(y) # Output: 192.0 If you found this output confusing, don\u0026rsquo;t worry. This notation (float.hex) looks like a \u0026ldquo;base salad\u0026rdquo; because it mixes Base 16 with powers of Base 2.\nLet\u0026rsquo;s detail the notation further. It follows the format: 0x[Hexadecimal Significand]p[Decimal Exponent]\nThe golden rule to decipher it is: $$ \\text{Value} = (\\text{Hexadecimal Number}) \\times 2^{(\\text{Exponent after the p})} $$Let\u0026rsquo;s dissect the two examples above to understand the math behind them:\nAnalyzing 0x1.5p+3 (which resulted in 10.5) The Significand (1.5): It is in Hexadecimal. The 1 is worth 1, but the 5 after the point is not \u0026ldquo;5 tenths\u0026rdquo;. In base 16, each place after the point divides by 16. So .5 hex means \\(5/16\\) or \\(0.3125\\) in decimal. Significand Value: \\(1 + 0.3125 = \\mathbf{1.3125}\\). The Exponent (p+3): The p indicates a power of 2. Exponent Value: \\(2^3 = \\mathbf{8}\\). Final Calculation: \\(1.3125 \\times 8 = \\mathbf{10.5}\\). Analyzing 0x0.3p10 (which resulted in 192.0) The Significand (0.3): Again, in Hexadecimal. The .3 hex means \\(3/16\\) or \\(0.1875\\) in decimal. The Exponent (p10): Exponent Value: \\(2^{10} = \\mathbf{1024}\\). Final Calculation: \\(0.1875 \\times 1024 = \\mathbf{192.0}\\). Why use this? This notation is useful for systems programmers because it defines the number exactly as it is stored in the hardware, without suffering the rounding errors typical of the Decimal → Binary conversion. It is an exact way to carry float numbers between different machines.\nThe Real Proof of the Precision Error # Remember that we discussed how \\(0.3_{10}\\) becomes a repeating fraction in binary? In Python, this produces the famous floating-point arithmetic \u0026ldquo;error\u0026rdquo;.\nSince the computer rounds the binary repeating fraction to the nearest representable value, the sum does not come out exact:\nval1 = 0.1 val2 = 0.2 total = val1 + val2 print(f\u0026#34;Result: {total}\u0026#34;) # Output: 0.30000000000000004 print(total == 0.3) # Output: False It is worth pausing here to understand where this error lives. The root of it all is that the computer\u0026rsquo;s number line is not continuous: it is made of spaced marks, and most real numbers fall between two marks. The figure below shows both sides of this: on top, how the marks are distributed (dense near zero, sparse far from it) and below, a zoom around \\(0.1\\), where you can see that the exact value simply has no mark:\nFloats do not cover the line: they are spaced marks, denser near zero. The exact 0.1 falls between two of them, and the computer stores the nearest one. The 0.1 you type, therefore, never reaches memory whole: what gets stored is the nearest mark. Add two of these approximations and the result escapes the mark that would correspond to 0.3: that is literally what the output above is showing.\nFor critical applications (such as financial systems) where this error is unacceptable, Python offers the decimal module, which does arithmetic in base 10 and is exact for decimal fractions that fit in the context\u0026rsquo;s precision (28 digits by default), at the cost of performance, avoiding the conversion to binary. Note the qualification: exact for decimal fractions, not for every division. Decimal(1) / Decimal(3) is still a repeating fraction, and is still rounded.\nfrom decimal import Decimal d1 = Decimal(\u0026#39;0.1\u0026#39;) d2 = Decimal(\u0026#39;0.2\u0026#39;) print(d1 + d2) # Output: 0.3 This behavior is not a bug or an implementation flaw of Python: it is a direct consequence of the base conversion we did by hand at the beginning of the article. The official documentation has a page dedicated just to this, Floating Point Arithmetic: Issues and Limitations, and it is worth reading.\nThe Costliest Case of \\(0.1\\) Not Being Representable On February 25, 1991, during the Gulf War, a Patriot anti-missile battery in Dhahran, Saudi Arabia, failed to intercept a Scud missile. The attack killed 28 American soldiers.\nThe cause was exactly what this article describes. The system\u0026rsquo;s internal clock counted tenths of a second, and the value \\(0.1\\) was stored in a 24-bit register, truncated, because \\(0.1\\) is a repeating fraction in binary. The error of each conversion was about \\(9.5 \\times 10^{-8}\\) seconds, negligible. But the battery had been on for about 100 hours, and the error accumulated to \\(0.34\\) seconds. For a Scud at 1,676 m/s, 0.34 seconds means more than half a kilometer of error in the predicted position, enough for the missile not to be where the system was looking.\nThe case is documented in the official GAO report (GAO/IMTEC-92-26) and is analyzed in detail on Professor Douglas Arnold\u0026rsquo;s page, at the University of Minnesota.\nStrategies to Work Around the Error # Besides using the Decimal module, there are two very common algorithmic approaches to deal with this limitation, depending on your problem domain:\nA. Working with Integers (Financial Approach)\nIn banking or e-commerce systems, we often avoid floats completely. The idea is to work with the smallest monetary unit (cents) as an integer. Instead of storing R$10.50, we store the integer 1050. Since integer math is exact, the rounding error disappears. We only divide by 100 when displaying the value on screen.\n# Approach with Floats (Dangerous for money) price_a = 0.10 price_b = 0.20 print(price_a + price_b) # 0.30000000000000004 # Approach with Integers (Safe) cents_a = 10 cents_b = 20 total = cents_a + cents_b print(f\u0026#34;R$ {total / 100:.2f}\u0026#34;) # R$ 0.30 B. Comparison by Tolerance (Scientific Approach)\nIn scientific or engineering contexts, we rarely check whether two real numbers are strictly equal (==). Instead, we check whether they are \u0026ldquo;close enough\u0026rdquo; to each other, within an acceptable margin of error called the tolerance, usually denoted by \\(\\epsilon\\) (the Greek letter epsilon).\nIn Python, you can do this manually or use the math.isclose function:\nimport math val1 = 0.1 + 0.2 expected = 0.3 # The wrong way if val1 == expected: print(\u0026#34;Equal\u0026#34;) # Never gets here # The manual way (defining a tolerance, e.g., 0.00001) tolerance = 1e-5 if abs(val1 - expected) \u0026lt; tolerance: print(\u0026#34;Equal (within the margin)\u0026#34;) # The Pythonic way (recommended) # Here we pass your variable to \u0026#39;abs_tol\u0026#39; (Absolute Tolerance) if math.isclose(val1, expected, abs_tol=tolerance): print(f\u0026#34;Equal (using math.isclose with tolerance of {tolerance})\u0026#34;) The math.isclose function is useful because it lets you define what \u0026ldquo;equal\u0026rdquo; means in two different ways. Understanding the difference between them is vital to avoid silent bugs:\nRelative Tolerance (rel_tol) - The Default What it is: A comparison based on percentage or magnitude. The default is 1e-9 (0.000000001), which means the numbers must agree up to the 9th significant digit, which is different from the 9th decimal place: for values on the order of millions, the 9th significant digit falls well before the point. How it works: the difference is compared with rel_tol multiplied by \\(\\max(|a|, |b|)\\), that is, against the larger of the two in absolute value, and not against the second argument. When to use: In most general cases. The Logic: An error of 1 meter is unacceptable if you are measuring a table (high relative error), but irrelevant if you are measuring the distance from the Earth to the Moon (low relative error). rel_tol adapts to the scale of the numbers. Absolute Tolerance (abs_tol) - The Fixed Value What it is: A fixed margin of error, like a ruler. If you set abs_tol=0.01, any difference smaller than 0.01 is accepted, no matter whether the numbers are huge or tiny. The default is 0.0. When to use: Mainly when comparing with Zero. The Logic: If you try to check math.isclose(0.0, 0.0000001), the relative tolerance will fail (since anything compared to zero is infinitely larger in percentage terms). In that case, you need to set an abs_tol to say: \u0026ldquo;I consider zero anything smaller than \\(10^{-7}\\)\u0026rdquo;. import math # Relative Tolerance example (Default) # Are 100 and 101 \u0026#34;close\u0026#34;? With 5% tolerance, yes. print(math.isclose(100, 101, rel_tol=0.05)) # True # Absolute Tolerance example (Zero) # Comparing with zero requires abs_tol a = 0.0 b = 1e-10 # 0.0000000001 print(math.isclose(a, b)) # False (The default fails with zero) print(math.isclose(a, b, abs_tol=1e-9)) # True (The difference is smaller than the fixed margin) The NumPy library, widely used in scientific contexts, also offers functions for comparison with tolerance, such as numpy.isclose and numpy.allclose, which are optimized for large arrays.\nSummary Table # Idea Rule Example from the article Fractional positional notation digits to the right of the point have weight \\(b^{-1}, b^{-2}, \\dots\\) \\(0.101_2 = 0.5 + 0.125 = 0.625\\) Decimal \\(\\to\\) another base successive multiplications; read the integer parts from top to bottom \\(0.625 \\to 0.101_2\\) When the process does not end the fraction becomes a repeating fraction in the target base \\(0.3 \\to 0.01001..._2\\), with a cycle Mixed number convert the two parts by different methods and join them \\(6.512 \\approx 110.10000_2\\) Price of truncating stopping early costs precision \\(110.10000_2\\) is worth \\(6.5\\): \\(0.012\\) was lost Grouping by power start at the point and move to the ends \\(0.\\mathtt{A7}_{16} = 0.10100111_2\\) Padding zeros on the left in the integer part, on the right in the fractional part \\(0.11011_2 = 0.\\mathtt{D8}_{16}\\) See the bits of a float struct.unpack('I', struct.pack('f', x)) 0.625 \\(\\to\\) 00111111001000000000000000000000 See the exact stored value Decimal(x), without quotes Decimal(0.3) has 54 digits Float in hexadecimal float.hex() and float.fromhex() (10.5).hex() \\(\\to\\) '0x1.5000000000000p+3' Money use integers in the smallest unit 1050 cents, not 10.50 Scientific comparison math.isclose, with rel_tol (scale) or abs_tol (zero) isclose(0.0, 1e-10, abs_tol=1e-9) is True Final Thoughts: The Problem of Precision # Throughout this article, you must have noticed that conversions such as \\(0.3_{10} \\to \\text{Binary}\\) result in infinite repeating fractions. Since the computer\u0026rsquo;s memory is finite, it cannot store \u0026ldquo;infinite bits\u0026rdquo;. It has to cut (truncate) or round this number at some point.\nThis means the computer almost never stores exactly the number \\(0.3\\), but rather the nearest representable value. And we don\u0026rsquo;t have to guess which one it is: the decimal module shows the stored number, digit by digit.\nfrom decimal import Decimal print(Decimal(0.3)) # without quotes: converts the float already in memory # Output: 0.299999999999999988897769753748434595763683319091796875 print(Decimal(0.1)) # Output: 0.1000000000000000055511151231257827021181583404541015625 Notice that Decimal('0.3'), with quotes, is the exact decimal number you wrote; Decimal(0.3), without quotes, is the translation of what the hardware actually stored. That is 54 digits to represent something you typed with a single digit after the point.\nIt is this small error, inherent to the conversion of fractional bases, that causes the \u0026ldquo;strange\u0026rdquo; behaviors in programming languages, such as 0.1 + 0.2 != 0.3.\nIn the article on IEEE 754, we will get into the anatomy of this standard, the universal system that defines how these bits (sign, exponent and significand) are organized to minimize these errors and allow scientific calculations.\nIf you want to go deeper, the classic reference on the subject is the article \u0026ldquo;What Every Computer Scientist Should Know About Floating-Point Arithmetic\u0026rdquo;, by David Goldberg (1991), dense, but it is the text everyone cites when the subject is floating-point error.\nSee you then!\n","date":"2026-04-01","externalUrl":null,"permalink":"/2026/04/real-numbers-base-conversion/","section":"Blog","summary":"Why doesn’t 0.1 plus 0.2 give 0.3? Because 0.1 is a repeating fraction in binary. We convert fractions between bases by successive multiplications, look at the bits of a float with struct, and see two practical ways out: integers for money and math.isclose for comparisons.","title":"Real Numbers and Floating Point: Base Conversion and Precision","type":"posts"},{"content":"","date":"2026-03-30","externalUrl":null,"permalink":"/tags/algebra-booleana/","section":"tags","summary":"","title":"álgebra booleana","type":"tags"},{"content":" In the previous articles, we explored computer arithmetic. We learned how computers add, subtract using two\u0026rsquo;s complement, and how they deal with the physical limits of memory (overflow). In the previous article, we briefly used the \u0026amp; operator to reveal the internal representation of negative numbers in Python. If you were curious about what that \u0026amp; really did and what else can be done with it, the time has come to find out.\nIn those arithmetic cases, the bits were treated together to represent a numeric value (positional weight). But sometimes we want to manipulate the structure of a number, not compute its value: turn on a specific \u0026ldquo;switch\u0026rdquo;, turn off a sensor, encrypt a message or check a user\u0026rsquo;s permissions.\nIt is time to set arithmetic aside for a while and enter the world of Bitwise Logic. Here, the processor does not see the number \u0026ldquo;5\u0026rdquo; or \u0026ldquo;10\u0026rdquo;, it sees a sequence of individual switches that can be on (1) or off (0).\nThe Connection with Propositional Logic # If you follow the blog, you may have already read our article on Logic for Programmers. There, we saw how to build truth tables to decide whether a complex proposition is True (T) or False (F).\nBitwise logic is the direct application of that concept, but on an industrial scale:\nTrue (T) becomes 1. False (F) becomes 0. Instead of comparing just two propositions (\u0026ldquo;The door is open\u0026rdquo; AND \u0026ldquo;The light is on\u0026rdquo;), the bitwise operator compares 8, 16, 32 or 64 pairs of propositions at once, in parallel.\nThe Big Warning: and Is Not \u0026amp; # Before we see the operators, we need to clear up a classic confusion in Python, the language most used on this blog. Unlike languages such as C or Java, Python is very verbal, using words like and, or and not. However, it also has the symbols \u0026amp;, | and ~.\nThey are not the same thing.\nIn the article on Short-Circuiting in Python, we explained that the logical operators (and, or) are \u0026ldquo;lazy\u0026rdquo;. They stop evaluating as soon as they find the answer.\nThe bitwise operators (\u0026amp;, |), on the other hand, are \u0026ldquo;tireless workers\u0026rdquo;. They evaluate every bit, without exception, and produce a new number as the result.\nTable of Differences # Feature Logical Operators Bitwise Operators Target Boolean expressions (True/False) or objects Integers (individual bits) Evaluation Short-circuit (lazy evaluation) Complete (eager evaluation) Result Returns one of the operands (e.g., True) Returns a new integer Example if age \u0026gt; 18 and has_license: masked = flags \u0026amp; 0xFF Careful in Python Never use \u0026amp; in place of and inside an if, unless you know exactly what you are doing.\n5 and 6 returns 6 (because 5 is truthy, it goes on and returns the last one). 5 \u0026amp; 6 returns 4 (because it intersects the bits of 101 and 110). They are completely different behaviors that can cause silent bugs.\nNow that we have separated flow logic from bit logic, let us meet the four fundamental tools: AND, OR, XOR and NOT.\nThe Mathematics of Bits: AND and OR # To understand bitwise, we need to stop looking at the decimal number (e.g., 12) and start looking at its binary representation. The operations happen vertically, aligning the corresponding bits of each number.\nAND (\u0026amp;) - The Intersection # The Bitwise AND operator follows a strict rule: the result bit will be 1 if, and only if, the bits of both operands are 1.\nThink of AND as a Filter or an Intersection.\n1 \u0026amp; 1 = 1 1 \u0026amp; 0 = 0 0 \u0026amp; 1 = 0 0 \u0026amp; 0 = 0 Note that Zero is dominant: anything combined with zero becomes zero. This property makes AND the perfect tool for clearing bits or checking values (a technique known as masking).\nPractical Example: 12 \u0026amp; 10 Let us compute 12 \u0026amp; 10.\n12 in binary is 1100. 10 in binary is 1010. We align the bits and apply the logic column by column: $$ \\begin{array}{r c c c c l} \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 (12) \\\\ \\\u0026 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 (10) \\\\ \\hline \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 (8) \\end{array} $$ Analysis:\nBit 3 (left): 1 \u0026amp; 1 results in 1. Bit 2: 1 \u0026amp; 0 results in 0. Bit 1: 0 \u0026amp; 1 results in 0. Bit 0: 0 \u0026amp; 0 results in 0. Result: 1000 (which is 8 in decimal).\nIn Python:\nprint(12 \u0026amp; 10) # 8 print(bin(12 \u0026amp; 10)) # 0b1000 OR (|) - The Union # The Bitwise OR operator is more permissive: the result bit will be 1 if at least one of the operand bits is 1.\nThink of OR as a Logical Sum or Union.\n1 | 1 = 1 1 | 0 = 1 0 | 1 = 1 0 | 0 = 0 Here, One is dominant: anything combined with 1 becomes 1. This property makes OR the ideal tool for setting bits (forcing a value to 1) without changing the others.\nPractical Example: 12 | 10 Let us compute 12 | 10.\n12 in binary is 1100. 10 in binary is 1010. We align the bits and apply the logic column by column: $$ \\begin{array}{r c c c c l} \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 (12) \\\\ \\| \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 (10) \\\\ \\hline \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 (14) \\end{array} $$Analysis:\nWherever there was at least one 1 bit in the inputs, the output was 1. Only the last column (0 | 0) resulted in zero. Result: 1110 (which is 14 in decimal).\nIn Python:\nprint(12 | 10) # 14 print(bin(12 | 10)) # 0b1110 Summary of the Logic # So you do not forget:\nAND (\u0026amp;): Used to Cut/Clear. If you want to guarantee that certain bits are zero, use AND with 0. OR (|): Used to Add/Set. If you want to guarantee that certain bits are one, use OR with 1. In the next section, we will see two other operators: XOR (the difference finder) and NOT (the total inversion).\nThe XOR and NOT Operators # While AND and OR are intuitive (they look like the intersection and union of sets), the next two operators have unique behaviors that are the basis of cryptography and integer arithmetic.\nXOR (^) - The Difference Detective # The Exclusive OR (XOR) is a common operator in cryptography and hash algorithms. The rule is: the result bit will be 1 if the operand bits are different.\n1 ^ 1 = 0 (Equal \\(\\to\\) 0) 0 ^ 0 = 0 (Equal \\(\\to\\) 0) 1 ^ 0 = 1 (Different \\(\\to\\) 1) 0 ^ 1 = 1 (Different \\(\\to\\) 1) Think of XOR as a Toggle.\nIf you XOR with 0, the original bit is kept (1^0=1, 0^0=0). If you XOR with 1, the original bit is flipped (1^1=0, 0^1=1). Its usefulness in cryptography comes from a much stronger property than \u0026ldquo;there are as many 0s as 1s in the result\u0026rdquo;: if the key bit is uniformly random, the output bit is uniformly random, no matter what the message bit was. Looking at the result tells you nothing about the original text, not even a little. This is the argument for the one-time pad, the classic example of a cipher with perfect secrecy, a result proved by Claude Shannon in 1949.\nPractical Example: 12 ^ 10 Let us compute 12 ^ 10. $$ \\begin{array}{r c c c c l} \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 (12) \\\\ \\text{\\textasciicircum} \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 (10) \\\\ \\hline \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 (6) \\end{array} $$ Analysis:\nBit 3: 1 and 1 are equal \\(\\to\\) 0. Bit 2: 1 and 0 are different \\(\\to\\) 1. Bit 1: 0 and 1 are different \\(\\to\\) 1. Bit 0: 0 and 0 are equal \\(\\to\\) 0. Result: 0110 (which is 6 in decimal).\nCryptographic curiosity: XOR is reversible. If you compute (A ^ B) ^ B, the result goes back to A.\nkey = 123 message = 456 encrypted = message ^ key print(encrypted) # 435 print(encrypted ^ key) # 456 (XOR with the same key undoes the operation) A homemade XOR cipher protects nothing The perfect secrecy of the previous paragraph holds under two strict conditions: the key must be random, as long as the message, and never reused. An XOR cipher with a short key repeated along the text (which is what almost everybody writes on the first attempt) falls to frequency analysis, without even needing to guess the key: positions separated by a multiple of the key length were encrypted with the same byte, which reduces the problem to a simple substitution cipher. A great bit exercise; a terrible choice for protecting anything real.\nThis reversibility yields a classic trick: swapping the contents of two variables without using a third one.\na, b = 5, 9 a ^= b # a now holds the \u0026#34;difference\u0026#34; between the two b ^= a # b receives the original value of a a ^= b # a receives the original value of b print(a, b) # 9 5 It works, but it is worth more as a demonstration of the property than as a technique. In Python, a, b = b, a is clearer and faster. And the trick hides a trap: if both names point to the same position (items[i] and items[j] with i == j), the first XOR zeroes the value and it never comes back.\nNOT (~) - The Total Inversion # The Bitwise NOT operator is the only one that operates on a single number (a unary operator). It simply flips all the bits: what is 0 becomes 1, and what is 1 becomes 0.\nVisually, it looks simple. But if you test it in Python, the result may confuse you:\nprint(~0) # -1 print(~10) # -11 Why is the result negative?\nHere the knowledge from our previous article on two\u0026rsquo;s complement comes in.\nPython treats integers as if they had \u0026ldquo;infinite bits\u0026rdquo; on the left. The positive number 10 is ...00001010. When we apply NOT (~), we flip all the bits, including the infinite zeros on the left. The number becomes ...11110101. An infinite sequence of 1s on the left indicates a negative number. Mathematically, in two\u0026rsquo;s complement systems, the operator ~x is equivalent to the one\u0026rsquo;s complement. The relation with the decimal value follows the formula:\nUnderstanding the Formula (~10) We want to compute ~10.\nBy the mathematical formula: -(10) - 1 = -11. By the bit view (in 8 bits for simplicity): 10 is 0000 1010.\nFlipping everything (~): 1111 0101.\nWhat is 1111 0101? Since the first bit is 1, it is negative.\nTo find the value, we apply the inverse rule (flip and add 1):\nFlip: 0000 1010 (10)\nAdd 1: 0000 1011 (11)\nSo the original value was -11.\nIn the next topic, we will see how to move these bits from side to side with the Shift operators.\nShift: The Arithmetic of Displacement # Imagine you have the decimal number 23. If you add a zero on the right (230), you effectively multiplied the number by 10 (the base of the system). If you remove the last digit (the 3), you divided by 10 (integer division, leaving 2).\nIn the binary system, the logic is identical, but the base is 2. In hardware and in compiled code, moving bits left or right tends to be cheaper than multiplying or dividing by powers of 2. In Python the gain is not guaranteed, and the reason to use shifting is clarity of intent.\nLeft Shift (\u0026lt;\u0026lt;) - The Multiplier # The Left Shift operator (x \u0026lt;\u0026lt; n) moves all the bits of x to the left by n positions.\nThe bits that \u0026ldquo;fall\u0026rdquo; off the left are discarded (in fixed-size types) or the number grows (in Python). Zeros are inserted in the vacant positions on the right. Mathematical rule: Shifting n bits to the left is equivalent to multiplying the number by \\(2^n\\).\n$$ x \\ll n \\iff x \\times 2^n $$ Example: 3 \u0026laquo; 2 The number 3 in binary (8 bits) is 0000 0011. Let us shift 2 places to the left.\nPush everything to the left. Fill the holes on the right with 0. $$ \\begin{array}{r c c l} \u0026 0000 \u0026 0011 \u0026 (3) \\\\ \\downarrow \u0026 \\text{Shift Left 2} \u0026 \u0026 \\\\ \u0026 0000 \u0026 11\\mathbf{00} \u0026 (12) \\end{array} $$ Check: \\(3 \\times 2^2 = 3 \\times 4 = 12\\). Right Shift (\u0026gt;\u0026gt;) - The Divider # The Right Shift operator (x \u0026gt;\u0026gt; n) moves the bits to the right.\nThe bits on the right \u0026ldquo;fall\u0026rdquo; into the abyss and are lost. What enters on the left? Here is the big technical detail. In low-level languages, there are two kinds of right shift:\nLogical Shift: Always fills with 0. Used for unsigned numbers. Arithmetic Shift: Fills with the value of the sign bit (the leftmost bit). If the number was negative (started with 1), it fills with 1s to keep the number negative. How does Python do it? Since Python handles signed numbers with arbitrary precision, the \u0026gt;\u0026gt; operator behaves as an Arithmetic Shift. It preserves the mathematical sign.\nMathematical rule: Shifting n bits to the right is equivalent to a Floor Division by \\(2^n\\).\n$$ x \\gg n \\iff \\lfloor x / 2^n \\rfloor $$With both operators introduced, it is worth seeing the two movements drawn side by side. Notice what comes in and what goes out in each case, because this asymmetry explains why one of them is reversible and the other is not:\nShifting left only creates zeros; shifting right discards bits, and what was discarded is exactly the remainder of the division. The \u0026lt;\u0026lt; pushes zeros in and, as long as there is room, nothing is lost. The \u0026gt;\u0026gt;, on the other hand, throws away the bits at the right end, and those bits were precisely the remainder of the division. That is why the right shift is equivalent to an integer division: the remainder does not go anywhere, it just vanishes.\nPositive vs. Negative Example Case 1: Positive (12 \u0026raquo; 2)\nBinary: 0000 1100\nShift right, 0 enters: 0000 0011\nResult: 3. (Math: \\(12 / 4 = 3\\)).\nCase 2: Negative (-12 \u0026raquo; 2)\nBinary (two\u0026rsquo;s complement): ...1111 0100\nShift right. Since the sign bit is 1, 1s enter from the left.\nResult: ...1111 1101\nThe result is -3. (Math: \\(-12 / 4 = -3\\)).\nThis guarantees that division by powers of 2 works correctly for both positive and negative numbers.\n\u0026gt;\u0026gt; is floor, not truncation The example above gives the impression that \u0026gt;\u0026gt; and \u0026ldquo;dividing by 4\u0026rdquo; are the same thing, because \\(-12\\) is a multiple of 4 and the result comes out round. Change the number to \\(-13\\) and the difference shows up: -13 \u0026gt;\u0026gt; 2 returns -4, whereas a division that truncates toward zero (like the / between integers in C) would return -3.\nThe reason lies in the formula itself, \\(\\lfloor x / 2^n \\rfloor\\): the floor rounds down, and for negative numbers \u0026ldquo;down\u0026rdquo; means away from zero. In Python this is consistent with //, which is also floor (-13 // 4 is -4). If you really need truncation, int(-13 / 4) returns -3.\nIn the next section, we will put together everything we have learned (AND, OR, NOT, SHIFT) to create practical tools called Bitmasks.\nBit Masks (Bitmasks) # Now that we know the operators, we can combine them to make precise changes to numbers. The technique of using an auxiliary number to \u0026ldquo;select\u0026rdquo; or \u0026ldquo;modify\u0026rdquo; specific bits of another number is called Masking.\nFor the examples below, imagine we want to manipulate the 3rd bit (from right to left, index 2). For that, we create a mask where only that bit is 1:\nMASK = 1 \u0026lt;\u0026lt; 2 # Result: 0b00000100 (Decimal 4) Checking a Bit (Check) # We want to know: \u0026ldquo;Is the 3rd bit on?\u0026rdquo;. We use AND (\u0026amp;). Since AND with 0 zeroes everything, if the result is nonzero, it means our target bit was on.\nflags = 0b10110111 is_on = (flags \u0026amp; MASK) != 0 # The result will be True if the 3rd bit is on (which is the case), False otherwise. Note the != 0, and not \u0026gt; 0. With the values of this example both work, but the previous article spent a good part of its time showing that integers can be negative. If flags is negative (something ~ produces easily), the result of \u0026amp; can be negative too, and a \u0026gt; 0 test would answer \u0026ldquo;bit off\u0026rdquo; for a bit that is on. Comparing with zero by inequality is always safe.\nSetting a Bit (Set) # We want to force the 3rd bit to be 1, without touching the others. We use OR (|). Since OR with 0 preserves the original value, and OR with 1 forces it to 1, our mask protects the rest and lights up the target.\nflags = flags | MASK # Now the 3rd bit is definitely 1. The others did not change. Clearing a Bit (Clear) # We want to force the 3rd bit to be 0. This is the most logically complex operation. We need an AND (to zero), but we need a mask that has 0 at the target and 1 everywhere else (to preserve the neighbors). How do we generate that? By inverting our original mask with NOT (~).\nThe Logic of Clear (Clearing a Bit) Original mask (MASK): ...0000 0100 Inverted mask (~MASK): ...1111 1011 Operation (flags \u0026amp; ~MASK): ...10110011 The target bit was zeroed, the others were preserved by the 1s of the mask.\nflags = flags \u0026amp; (~MASK) Toggling a Bit (Toggle) # We want to invert the current state: if it was on, turn it off; if it was off, turn it on. We use XOR (^), since XOR with 1 always flips the bit.\nflags = flags ^ MASK The four operations form a closed set, and it is useful to see them together after having seen them one by one. The figure below repeats the same target bit in the four cases, with its column highlighted, so that we can compare what each operator does to the neighbors:\nThe four mask operations on the same target bit: AND checks, OR lights up, AND with the inverted mask clears and XOR flips. Notice the pattern: the target bit changes in all of them, and the other seven columns stay intact in all of them. That is the point of masking: changing one position without touching any other.\nTwo Functions That Save Work # Python has two built-in operations that show up all the time in real bit-manipulation code and that save you from writing loops by hand:\nflags = 0b10110111 flags.bit_count() # 6 (how many bits are on; Python 3.10+) flags.bit_length() # 8 (how many bits are needed to represent the value) bit_count() is what the literature calls popcount, and it answers in one call questions like \u0026ldquo;how many permissions does this user have?\u0026rdquo; or \u0026ldquo;how many items are checked?\u0026rdquo;. bit_length() answers \u0026ldquo;where is the most significant bit that is on?\u0026rdquo;, useful for knowing how many bits a value really occupies.\nIf this kind of manipulation interested you, the practical reference on the subject is Bit Twiddling Hacks, by Sean Eron Anderson, a collection of bit tricks for tasks that, at first sight, would seem to require loops and conditionals.\nReal Example: Permissions System # Let us see where this is used in practice. Operating systems (like Linux) use bits to control file permissions. Let us simulate this in Python.\nImagine we have 3 possible permissions, represented by bits:\nRead (READ): bit 2 (value 4) Write (WRITE): bit 1 (value 2) Execute (EXEC): bit 0 (value 1) # Defining the constants (MASKS) READ = 0b100 # 4 WRITE = 0b010 # 2 EXEC = 0b001 # 1 # Scenario 1: Creating a user with Read and Execute permission # We use OR (|) to combine the permissions user_permissions = READ | EXEC print(bin(user_permissions)) # Output: 0b101 (Decimal 5) # Scenario 2: Can the user write? # We use AND (\u0026amp;) to check if (user_permissions \u0026amp; WRITE): print(\u0026#34;Write access granted\u0026#34;) else: print(\u0026#34;Write access denied\u0026#34;) # It will land here # Scenario 3: Adding write permission dynamically user_permissions = user_permissions | WRITE print(bin(user_permissions)) # Now the value is 0b111 (7) - Full permission (rwx) # Scenario 4: Revoking execute permission # We use AND with NOT user_permissions = user_permissions \u0026amp; (~EXEC) print(bin(user_permissions)) # Now the value is 0b110 (6) - Read and Write only Notice how we managed to store 3 independent boolean pieces of information in a single integer variable, using only 3 bits of information.\nHere is a precision that often goes unnoticed: none of this takes up \u0026ldquo;3 bits of memory\u0026rdquo;. On most current architectures, the smallest addressable unit of memory is the byte, and a Python int, with all the arbitrary-precision machinery it carries, takes up 28 bytes (sys.getsizeof(5) shows this). What fits in 3 bits is the information. The real savings appear when these bits are packed by the millions, in network protocols, file formats and compact data structures.\nWhere the 755 in chmod Comes From The three bits we have just used (read, write, execute) are exactly the Unix ones, and now we can see why the permissions command shows up in octal. A file\u0026rsquo;s permissions are nine bits, in three groups of three: owner, group and others. And three bits is precisely what one octal digit represents. That is, chmod 755 is the octal notation for 111 101 101:\nGroup Octal digit Bits Meaning Owner 7 111 read, write, execute Group 5 101 read and execute Others 5 101 read and execute That is why the first article of this series devoted a whole section to octal: it is not a historical curiosity, it is the notation that makes nine permission bits readable at a glance. More details at file-system permissions.\nSummary Table # Operation What it does Example from the article \u0026amp; (AND) bit 1 only if both are 1 12 \u0026amp; 10 = 8 | (OR) bit 1 if either is 1 12 | 10 = 14 ^ (XOR) bit 1 if they differ 12 ^ 10 = 6 ~ (NOT) flips all bits: \\(\\sim x = -x - 1\\) ~10 = -11 \u0026lt;\u0026lt; shifts left: multiplies by \\(2^n\\) 3 \u0026lt;\u0026lt; 2 = 12 \u0026gt;\u0026gt; shifts right: \\(\\lfloor x / 2^n \\rfloor\\) -12 \u0026gt;\u0026gt; 2 = -3, but -13 \u0026gt;\u0026gt; 2 = -4 and / or operate on truth values, not on bits 5 and 6 returns 6, not 4 Check bit (flags \u0026amp; MASK) != 0 bit 2 of 0b10110111 is on Set bit flags | MASK preserves the neighbors Clear bit flags \u0026amp; ~MASK 0b10110111 \u0026amp; ~0b100 = 0b10110011 Toggle bit flags ^ MASK XOR with 1 always flips Create mask 1 \u0026lt;\u0026lt; n 1 \u0026lt;\u0026lt; 2 is 0b100 Count bits on x.bit_count() 0b10110111 has 6 Size in bits x.bit_length() 0b10110111 takes up 8 Undo XOR (A ^ B) ^ B == A basis of the cipher and of the swap without an extra variable Conclusion # Entering the Bitwise world is like looking at the Matrix: beneath the numbers appears the binary structure that supports them.\nAlthough Python is a high-level language, understanding \u0026amp;, |, ^ and \u0026lt;\u0026lt; is useful for dealing with network protocols, image processing, cryptography and algorithm optimization.\nAnd if you do not use Python, these concepts are universal and apply to practically every programming language.\nNow you know that:\nBitwise is not Logical: \u0026amp; is not and. Masks are the key: Use AND to read/clear and OR to write. Bits are versatile: A single integer can hold dozens of independent \u0026ldquo;switches\u0026rdquo;. Ready to practice? Try writing code that encrypts a string using XOR and then decrypts it using the same key, remembering the warning from above: this is a bit-manipulation exercise, not real protection.\nWith that, we close the chapter on integers in this series. Over the last few articles, we saw how to represent, convert, operate on and manipulate integers in different bases, including in the raw bit structure. Now it is time to take the next leap: how does the computer handle fractional numbers? Why is 0.1 + 0.2 not exactly 0.3 in Python? The answer begins with something you already master (base conversion), but now applied to the part after the decimal point.\nSee you next time!\n","date":"2026-03-30","externalUrl":null,"permalink":"/2026/03/bitwise-operators-masks-python/","section":"Blog","summary":"In Python, 5 and 6 returns 6, and 5 \u0026 6 returns 4. We explain the difference and the bitwise operators AND, OR, XOR, NOT and the shifts, used in masks that check, set, clear and toggle bits. In the end, we see where the 755 in chmod comes from.","title":"Bit-Level Logic: Bitwise Operators and Masks in Python","type":"posts"},{"content":"","date":"2026-03-30","externalUrl":null,"permalink":"/tags/bitwise/","section":"tags","summary":"","title":"bitwise","type":"tags"},{"content":"","date":"2026-03-30","externalUrl":null,"permalink":"/tags/boolean-algebra/","section":"tags","summary":"","title":"Boolean algebra","type":"tags"},{"content":"","date":"2026-03-30","externalUrl":null,"permalink":"/tags/operadores/","section":"tags","summary":"","title":"operadores","type":"tags"},{"content":"","date":"2026-03-30","externalUrl":null,"permalink":"/tags/operators/","section":"tags","summary":"","title":"operators","type":"tags"},{"content":"","date":"2026-03-27","externalUrl":null,"permalink":"/tags/algorithms/","section":"tags","summary":"","title":"algorithms","type":"tags"},{"content":"","date":"2026-03-27","externalUrl":null,"permalink":"/tags/algoritmos/","section":"tags","summary":"","title":"algoritmos","type":"tags"},{"content":" In the previous article, we learned to add and subtract integers the way we do on paper, and we ended with a question: does the hardware do the same, borrowing from neighboring bits? And what if I told you that the computer, in fact, does not subtract at all?\nAs we saw at the end of that article, a subtractor circuit costs about the same as an adder; what makes no sense is keeping the two side by side doing almost identical work. For a single circuit to get the job done, computer engineering developed a method to turn subtractions into additions. Instead of computing \\(A - B\\), the computer computes \\(A + (-B)\\).\nBut how do we represent a negative number using only zeros and ones? If we have just two symbols, where do we put the \u0026ldquo;minus\u0026rdquo; sign? This is where Two\u0026rsquo;s Complement comes in, the fundamental basis of integer arithmetic in modern processors.\nThe Idea of the Complement # Before thinking about bits, let us analyze the problem in our ordinary decimal base. Imagine a very simple calculator with a display of only 4 digits. The largest number it can show is 9999.\nIf we add 1 to that value (9999 + 1), the mathematical result would be 10000. But since the display only has room for 4 digits, the leftmost 1 disappears (it overflows past the limit of the display), and the screen shows only 0000.\nNow imagine the opposite path. If the screen shows 0000 and we try to subtract 1, the calculator wraps around the other way and shows 9999.\nIn this limited system, the number 9999 behaves mathematically as if it were -1. Shall we test it? If we add 5 to 9999 (our \u0026ldquo;fake -1\u0026rdquo;): \\[ 5 + 9999 = 10004 \\]Discarding the first digit, which does not fit on the display, leaves 0004. That is, \\(5 + (-1) = 4\\). The computation worked perfectly using only addition!\nThis is the Radix Complement: we use the \u0026ldquo;large\u0026rdquo; numbers at the top of the count to represent the negatives.\nLet us now formalize this idea.\nThe complement of a number \\(N\\) in a given base \\(b\\) is the difference between the next power of \\(b\\) and the number, in that order: the power on top, the number below.\nComplement in base 10 The base 10 complement of a number \\(N\\) is given by \\(10^d - N\\), where \\(d\\) is the number of digits of the system. Let us represent the equivalent of \u0026ldquo;-734\u0026rdquo; in a 3-digit system, the smallest one that holds 734: $$ \\begin{array}{r r r r} 1 \u0026 0 \u0026 0 \u0026 0\\\\ -{} \u0026 7 \u0026 3 \u0026 4\\\\ \\hline \u00262 \u0026 6 \u0026 6 \\end{array} $$ The number 266 is the \u0026ldquo;complement\u0026rdquo; of 734. Adding 266 will have the same effect as subtracting 734 (ignoring the overflow, that is, the carry that goes past the limit of the display).\nThe size of the system is a choice, and it must be respected from beginning to end: on the 4-digit display from the start of the article, the complement of 734 would be \\(10^4 - 734 = 9266\\), and it is 9266 that would behave as \\(-734\\) there. The reasoning is the same; only the number of places in the system changes. We will visualize this a little later in this article. First, we need to understand how negative numbers are represented in binary.\nSigned Numbers in Binary # In the computer, we have a fixed number of bits (usually 8, 16, 32, or 64). To tell positives from negatives, we use the Most Significant Bit (the leftmost bit, called the MSB) as the Sign Bit.\nA 0 at the start indicates a Positive number. A 1 at the start indicates a Negative number. So far, so good. The question that remains is: what do we do with the other bits? Reserving the MSB for the sign is only half of the decision, and historically there have been three different answers for the other half (signed number representations).\nScheme How it writes \\(-5\\) in 8 bits How many zeros exist Does the same adder work for adding and subtracting? Sign-magnitude 10000101 (sign bit + the usual 5) two: 00000000 and 10000000 no One\u0026rsquo;s complement 11111010 (flip the bits) two: 00000000 and 11111111 no, it needs an extra step in the addition Two\u0026rsquo;s complement 11111011 (flip and add 1) just one yes The first two are far more intuitive for a human (sign-magnitude is literally \u0026ldquo;write the sign and then the usual number\u0026rdquo;) and yet they lost the contest. The reason has a name: double zero. In both schemes, \\(+0\\) and \\(-0\\) are different bit patterns that mean exactly the same thing. This wastes one of the available combinations and, worse, forces the hardware to check two patterns every time someone writes if (x == 0). Add to that the extra circuitry both require in the addition, and the contest settles itself.\nTwo\u0026rsquo;s complement has a single zero, needs no adjustment when adding, and is what practically every processor uses today. It charges a price, which we will meet again later when discussing the limits of the representation: one combination is left over on the negative side, precisely the one that \\(-0\\) would occupy in the other schemes.\nFor example, in an 8-bit system:\n00000101 represents +5. 11111011 represents -5 (in two\u0026rsquo;s complement). Why is 11111011 equal to -5? Let us find out. Using positional notation:\nInterpreting negative numbers in two\u0026rsquo;s complement Let us interpret the binary number 11111011 as a signed integer (8 bits). Expanding: $$ 1 \\times 2^7 + 1 \\times 2^6 + 1 \\times 2^5 + 1 \\times 2^4 + 1 \\times 2^3 + 0 \\times 2^2 + 1 \\times 2^1 + 1 \\times 2^0 $$ Computing the values: $$ 128 + 64 + 32 + 16 + 8 + 0 + 2 + 1 = 251 $$ Since the leftmost bit is 1, we know the number is negative. To find its real value, we subtract \\(256\\) (which is \\(2^8\\), the next power of 2): $$ 251 - 256 = -5 $$ Therefore, 11111011 represents -5 in two\u0026rsquo;s complement. This method works for any number of bits.\nAnother way is to consider positional notation with a negative sign bit: $$ -1 \\times 2^7 + 1 \\times 2^6 + 1 \\times 2^5 + 1 \\times 2^4 + 1 \\times 2^3 + 0 \\times 2^2 + 1 \\times 2^1 + 1 \\times 2^0 $$ Computing the values: $$ -128 + 64 + 32 + 16 + 8 + 0 + 2 + 1 = -5 $$ Both ways lead to the same result.\nThis second reading, the negative-weight one, becomes much more concrete when drawn. In the ruler below, the eight bits appear with the weight of each position on top and the corresponding term below, exactly as you would do with an unsigned number. The only difference is at the left end: that weight comes with its sign flipped, and that is all that separates \\(251\\) from \\(-5\\).\nThe same eight bits, with a single weight flipped in sign: it is the bit at the end that turns the integer negative. Two\u0026rsquo;s Complement (Base 2) # The base 2 complement of a number \\(N\\) is mathematically given by \\(2^d - N\\).\nBase 2 complement (Formal Mathematical Computation) Let us find the negative of \\(0101_2\\) (5 in decimal) in a 4-bit system. The computation is \\(2^4 - 5\\), that is, \\(10000_2 - 0101_2\\).\nRemember: in the formal algorithm already presented in this series of articles, when the top digit is smaller, we add the base (2) to it and add 1 to the bottom digit of the next column. $$ \\begin{array}{r r r r r r} \u0026 1 \u0026 0^{\\scriptscriptstyle+2} \u0026 0^{\\scriptscriptstyle+2} \u0026 0^{\\scriptscriptstyle+2} \u0026 0^{\\scriptscriptstyle+2} \\\\ -\u0026 0^{\\scriptscriptstyle+1} \u0026 0^{\\scriptscriptstyle+1} \u0026 1^{\\scriptscriptstyle+1} \u0026 0^{\\scriptscriptstyle+1} \u0026 1 \\\\ \\hline \u0026 0 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \\end{array} $$ Step by step:\nCol 0: \\(0 \u003c 1\\). We add the base to the top (\\(0+2=2\\)). \\(2-1=\\mathbf{1}\\). We compensate (+1) in the next one. Col 1: Top \\(0\\), Bottom \\(0+1=1\\). \\(0 \u003c 1\\). We add the base (\\(2\\)). \\(2-1=\\mathbf{1}\\). We compensate. Col 2: Top \\(0\\), Bottom \\(1+1=2\\). \\(0 \u003c 2\\). We add the base (\\(2\\)). \\(2-2=\\mathbf{0}\\). We compensate. Col 3: Top \\(0\\), Bottom \\(0+1=1\\). \\(0 \u003c 1\\). We add the base (\\(2\\)). \\(2-1=\\mathbf{1}\\). We compensate. Col 4: Top \\(1\\), Bottom \\(0+1=1\\). \\(1-1=\\mathbf{0}\\). Result: \\(1011_2\\) (the 4 useful bits).\nDoing this subtraction is laborious. Fortunately, there is a much simpler algorithm used by hardware and by programmers: the \u0026ldquo;Flip and Add 1\u0026rdquo; technique.\nTechnique: Flip and Add 1 # This technique is based on the One\u0026rsquo;s Complement (just flipping the bits) and then adding 1 to arrive at the Two\u0026rsquo;s Complement.\nLet us find the two\u0026rsquo;s complement of the number 5 (0101) in a 4-bit system:\nTake the positive number: 0101 Flip all the bits (0 becomes 1, 1 becomes 0): 1010 Add 1 to the result. Base 2 complement (Flip and Add 1) After flipping the bits of 0101, we got 1010. Now, the work is just adding 1. $$ \\begin{array}{r r r r r} \u0026 1 \u0026 0 \u0026 1 \u0026 0 \\\\ +{} \u0026 \u0026 \u0026 \u0026 1 \\\\ \\hline \u0026 1 \u0026 0 \u0026 1 \u0026 1 \\end{array} $$ The result 1011 represents -5 in Two\u0026rsquo;s Complement.\nWhy does adding 1 work? # You may be wondering: why does flipping the bits and adding 1 give exactly the mathematical complement (\\(2^d - N\\), in a system of \\(d\\) bits)?\nThe explanation lies in the relation between a sequence full of \u0026ldquo;ones\u0026rdquo; and the next power of 2:\nFlipping the bits is mathematically equivalent to subtracting the number from a sequence made only of ones (this is called the One\u0026rsquo;s Complement). Example in 4 bits: Flipping \\(x\\) is equal to \\(1111_2 - x\\). The formal definition of Two\u0026rsquo;s Complement is to subtract from the next power of the base (\\(10000_2\\)). That is: \\(10000_2 - x\\). Since we know that \\(10000_2\\) is exactly \\(1111_2 + 1\\), we can substitute in the formula:\n$$ \\underbrace{(10000_2 - x)}_{\\text{Formal Definition}} = \\underbrace{(1111_2 + 1) - x}_{\\text{Substitution}} = \\underbrace{(1111_2 - x)}_{\\text{Flipped Bits}} + 1 $$In short: flipping the bits takes us \u0026ldquo;almost\u0026rdquo; there (up to the maximum value of the base minus 1). Adding 1 completes the difference to reach the correct power of 2. That is why the algorithm works perfectly.\nRule of Thumb (The Visual Shortcut) # For humans, there is an even faster way to do this \u0026ldquo;in your head\u0026rdquo;, with no computation:\nRule of thumb for two\u0026rsquo;s complement To obtain the two\u0026rsquo;s complement of a number directly, scan the number from right to left, repeating the zero digits until you find the first digit 1, which must be kept. From there on, all the digits to the left of that first 1 must be flipped.\nExamples of applying the rule:\nNumber: \\(011\\mathbf{1}00\\)\nKeep the ending 100. Flip the beginning 011 \\(\\to\\) 100. Result: \\(100\\mathbf{1}00\\) Additional examples:\n\\(1010 \\to 0110\\) \\(11001 \\to 00111\\) \\(111000 \\to 001000\\) \\(1100110 \\to 0011010\\) The Trick: Subtraction via Addition # Now that we know how to convert a number to negative (two\u0026rsquo;s complement), subtraction becomes trivial. To compute \\(A - B\\), the computer simply computes \\(A + \\text{Complement}(B)\\).\nSubtraction in base 10 using the radix complement Subtraction \\(913 - 734 = 179\\)\nThe base 10 complement of 734 is \\(1000 - 734 = 266\\). Thus, the subtraction \\(913 - 734\\) can be turned into an addition: \\(913 + 266 = 1179\\).\nNotice that the final result has one more digit than the original numbers (\\(1\\)). That digit is the carry that overflows the capacity of the representation. It is ignored to obtain the correct result (\\(179\\)).\nSubtraction in base 2 using the radix complement Let us compute \\(25 - 19\\) in binary (using 5 bits for simplicity). Then: \\(25 = 11001_2\\) and \\(19 = 10011_2\\)\nWe want to do: \\(11001_2 - 10011_2\\).\nStep 1: Find the two\u0026rsquo;s complement of 19 (\\(10011\\)). Using the rule of thumb: keep the last 1, flip the rest to the left \\(\\to\\) \\(\\mathbf{01101}\\).\nStep 2: Add 25 and (-19). We turn the subtraction into an addition: \\(11001 + 01101\\).\nWriting the operation vertically, we have: $$ \\begin{array}{c c c c c c} {} \u0026 \\scriptscriptstyle 1 \u0026 \u0026 \u0026 \\scriptscriptstyle 1 \u0026 \\\\ {} \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 1 \\\\ {} + \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 1 \\\\ \\hline 1) \u0026 0 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \\end{array} $$The \u0026ldquo;1)\u0026rdquo; bit on the left is the carry out of the number of bits. It is discarded. Final result: \\(00110_2\\) (which is 6 in decimal). Correct!\nLimits of the Representation (Range) # Before we talk about computation errors, we need to understand the limits of the playing field. Since the number of bits is finite, there is a maximum and a minimum value we can write. If we try to go past those limits, the number \u0026ldquo;does not fit\u0026rdquo;.\nThe range of values depends on two factors:\nHow many bits (\\(N\\)) we have available (8, 16, 32, 64\u0026hellip;). Whether we are using Unsigned or Signed numbers. Unsigned Numbers # Here, all the bits are used to represent the magnitude (the value) of the number. There are no negative numbers.\nMinimum: It is always 0 (all bits zero). Maximum: All bits are 1. The formula is \\(2^N - 1\\). Example with 8 bits:\nFrom \\(0\\) to \\(2^8 - 1\\) From 0 to 255. Signed Numbers (Two\u0026rsquo;s Complement) # Here, we split the possible combinations in half: half for the positives (and zero) and half for the negatives. The leftmost bit (MSB) defines the sign, leaving \\(N-1\\) bits for the value.\nThe formula changes slightly:\nMinimum (Negative): \\(-2^{N-1}\\) Maximum (Positive): \\(2^{N-1} - 1\\) Why is the maximum one less than the absolute value of the minimum? Because zero takes up a place on the \u0026ldquo;positive\u0026rdquo; side (where the sign bit is 0). In 8 bits, we have 256 combinations:\n128 negative combinations (from -1 to -128). 128 non-negative combinations (0 to +127). There is an image that arranges everything we have seen so far in a single drawing, and it is worth keeping. Instead of thinking of the numbers as a line running from \\(-8\\) to \\(+7\\), think of them as a clock: the bit combinations laid out in a circle, 0000 at the top, adding 1 being one step clockwise. The figure uses 4 bits because 16 positions fit on the page; in 8 bits it is the same clock, just with 256 marks.\nThe 16 combinations of 4 bits in a circle: adding 1 means turning one step clockwise, and the seam between +7 and −8 is where overflow happens. The clock answers four questions at once that this article treats in separate sections:\nWhy the complement works. Walking 5 steps backward from zero lands in the same place as walking 11 forward, because the circle closes. That is why subtracting is adding the complement. Why the range is asymmetric. There is only one mark for zero, at the top. Since the 16 positions split into two equal halves, one combination is left over on the negative side, and that is why we go from \\(-8\\) to \\(+7\\). What overflow is. It is crossing the seam by accident: adding two positives and landing on the red side of the circle. Why \\(-8\\) has no opposite. \\(+8\\) simply does not exist on the clock. The position where it would land is occupied by \\(-8\\) itself. We will come back to each of these points later, but keep the drawing.\nReference Table # See how the range changes drastically depending on the interpretation:\nBits (\\(N\\)) Type Minimum Value Maximum Value 4 bits Unsigned 0 15 4 bits Signed -8 +7 8 bits Unsigned 0 255 8 bits Signed -128 +127 16 bits Unsigned 0 65,535 16 bits Signed -32,768 +32,767 Thus, if you have an 8-bit signed system and try to add \\(100 + 30\\), the mathematical result (130) is greater than the allowed limit (+127). The computer does not have enough bits to represent that value, causing what we call Overflow. We will understand this better later on. But first, an important detail.\nThe Minimum Value Paradox (A Dangerous \u0026ldquo;Gotcha\u0026rdquo;) # If you look at the table above carefully, you will notice an asymmetry: in 8 bits, we go from -128 to +127.\nThis creates a curious problem: the number -128 has no positive opposite. The value +128 simply does not fit in 8 signed bits.\nThis asymmetry is the price of the single zero. Since two\u0026rsquo;s complement does not spend a bit pattern on \\(-0\\), one combination is left over on the negative side, and that combination is \\(-128\\). In the schemes that lost the contest back there, the range was symmetric, but at the cost of having two zeros.\nWhat happens if we apply the \u0026ldquo;Flip and Add 1\u0026rdquo; rule precisely to -128 (10000000)? Let us see:\nOriginal number: 10000000 (-128) Flip the bits: 01111111 (+127) Add 1: $$ \\begin{array}{r r} \u0026 01111111 \\\\ +{} \u0026 1 \\\\ \\hline \u0026 10000000 \\end{array} $$Result: 10000000. We are back to -128!\nThis means that, computationally, the opposite of -128 is itself. This behavior causes silent bugs in programming.\nBe Careful in Programming In C and C++, computing the absolute value abs() or negating -x of the smallest possible integer is undefined behavior: the language standard does not say what happens, and the compiler may optimize the code assuming this situation never occurs.\nJava has no undefined behavior, but that does not save it from the problem: Math.abs(Integer.MIN_VALUE) is specified and returns Integer.MIN_VALUE itself, a negative value coming out of an absolute value function. The bug is the same in both cases; the difference is that in one it is unpredictable and in the other it is documented.\nHere we talk about 8 bits, but the same holds for 16, 32, or 64 bits. There will always be a negative number with no positive opposite.\nOverflow (Capacity Overflow) # In the examples above, we discarded the last carry and the result was right. But that is not always the case.\nOverflow occurs when the result of an operation is too large (positively or negatively) to fit in the available number of bits.\nRemember: in an 8-bit signed system, we can represent numbers from -128 to +127. If you try to add \\(100 + 30\\), the result (130) does not fit, and it will invade the sign bit, looking like a negative number.\nOverflow detection follows two rules. The \u0026ldquo;human\u0026rdquo; (logical) rule and the \u0026ldquo;technical\u0026rdquo; (hardware) rule.\nHuman Rule: If we add two Positive numbers and the result is Negative, there was Overflow. (The same holds for Negative + Negative = Positive). Technical Rule: Overflow occurs if the carry going into the sign bit is different from the carry coming out of it. There is a carry propagated into the sign bit with no carry coming out of it. There is a carry propagated out of the sign bit without it having received a carry. Put another way: in two\u0026rsquo;s complement, overflow occurs if the carry-in (the carry that enters the sign bit) is different from the carry-out (the carry that leaves the sign bit).\nLet us see this happening in practice with 8-bit numbers:\nOverflow - Positive + Positive becomes Negative Let us add \\(88 + 46\\). The mathematical result would be 134. But in 8 signed bits, the maximum is 127. It will go wrong.\nLook at the Sign Bit (the leftmost one, bit 7). $$ \\begin{array}{c c c c c c c c c} {}\u0026\\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 \u0026 \u0026 \u0026 \\\\ {} \u0026 0 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \\\\ {} + \u0026 0 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 0 \\\\ \\hline {} \u0026 \\textcolor{red}{1} \u0026 0 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \\end{array} $$ Here the highlighted \\(\\textcolor{red}{1}\\) is the sign bit of the result: it is within the 8 bits, in the column of bit 7.\nAnalysis:\nLook at the sign bit (first column). A \u0026ldquo;carry\u0026rdquo; came in (it received 1 from the previous positions). But no \u0026ldquo;carry\u0026rdquo; went out of it, beyond the computation. It came in and did not go out? Overflow! The binary result 10000110 starts with 1, indicating a negative number (-122). This is obviously an error, since we added two positives.\nNow let us see the opposite case, with two negative numbers overflowing the lower limit (-128).\nOverflow - Negative + Negative becomes Positive Let us do \\( -76 + (-68) \\). The mathematical result would be -144. The lower limit is -128. It will go wrong.\nWe represent them in two\u0026rsquo;s complement and add: $$ \\begin{array}{c c c c c c c c c} \\scriptscriptstyle 1 \u0026 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 \u0026 \\\\ {} \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\\\ {} + \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \\\\ \\hline \\textcolor{red}{1} \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \\end{array} $$ Careful: the highlighted \\(\\textcolor{red}{1}\\) in this example is not the sign bit. It is the carry that came out of the computation, in an extra column to the left of the 8 bits, and it will be discarded. The sign bit of the result is the 0 right next to it.\nAnalysis:\nLook at the sign bit. It added \\(1 + 1\\), generating 0 and propagating a \u0026ldquo;carry\u0026rdquo; out (the red \\(\\textcolor{red}{1}\\)). But it received no \u0026ldquo;carry\u0026rdquo; from the previous column (see the empty space above the second column). It went out and did not come in? Overflow! The result 01110000 starts with 0, indicating a positive number (+112). Error detected.\nWhen Overflow Leaves the Paper # Overflow is behind some of the most expensive failures ever recorded in software.\nAriane 5, flight 501 On June 4, 1996, the Ariane 5 rocket exploded 39 seconds after liftoff, on its maiden flight. The cause: the inertial reference system converted the horizontal velocity from a 64-bit floating-point number to a 16-bit signed integer. On the Ariane 4, that value had always fit. The Ariane 5 was faster, the number did not fit, and the conversion overflowed.\nThe software handled the error by shutting down the inertial reference system (IRS) itself, and the backup IRS, which ran exactly the same code, had shut down for the same reason about 72 milliseconds earlier. With no attitude reference, the rocket turned, broke apart under aerodynamic pressure, and triggered its self-destruct. The loss is estimated at US$370 million, caused by a piece of code that did not even need to be active at that point of the flight.\nNot every case is tragic. Two well-known examples of the same phenomenon:\nPac-Man, level 256. The level counter had 8 bits, that is, it went from 0 to 255. On reaching level 256, the computation that draws the screen overflowed the counter and half of the maze turned into graphical garbage. It became known as the kill screen: the level is impossible to pass, and for decades it was the absolute ceiling of the game. The YouTube counter, in 2014. The view count was stored in a 32-bit signed integer, whose limit is \\(2^{31} - 1 = 2{,}147{,}483{,}647\\). When the video of Gangnam Style passed that mark, Google migrated the field to 64 bits, and made an official joke about it. With 64 bits, the new ceiling is over 9 quintillion. In all three cases the mechanism is the same one you just saw in the 8-bit computations above. What changes is the number of bits and, with it, the size of the damage.\nBonus Tip: Spotting Negatives in Hexadecimal # In practice, programmers rarely look at sequences of zeros and ones. The most common representation in debuggers and memory editors is Hexadecimal.\nHow can you tell whether a Hex number is negative (in two\u0026rsquo;s complement) at a glance?\nThe rule is universal, no matter whether the number has 8, 16, 32, or 64 bits: just look at the first digit on the left. If that digit represents a binary value that starts with 1 (that is, \\(\\ge 8\\)), the whole integer is negative.\nThe \u0026ldquo;Negative Digits\u0026rdquo; in Hexadecimal are: 8, 9, A, B, C, D, E, F\nExamples:\n8 bits: 0xFF \\(\\to\\) Starts with F. Negative (-1). 16 bits: 0x8000 \\(\\to\\) Starts with 8. Negative (-32,768). 32 bits: 0xC0000005 \\(\\to\\) Starts with C. Negative. Any size: 0x7FFF... \\(\\to\\) Starts with 7. Positive. If you are debugging code and see a (signed) integer variable starting with F, know immediately that it is negative. That is all: the first digit tells the sign, not the value. 0xF0000000 also starts with F and is worth \\(-268{,}435{,}456\\), which is quite different from \\(-1\\).\nThere is, however, one unmistakable special case: when all the digits are F, as in 0xFFFFFFFF (32 bits), then the value is indeed -1.\nIt may seem strange that the visually \u0026ldquo;largest\u0026rdquo; number (all filled with F or 1) represents -1. But we can prove it by applying the \u0026ldquo;Flip and Add 1\u0026rdquo; rule we just learned.\nIf we want to know the value of 0xFFFFFFFF (32 bits), we take the reverse path:\nConvert to Binary: A sequence of thirty-two 1s (1111...1111). Flip the bits: All the 1s become 0s (0000...0000). Add 1: $$ 0 + 1 = 1 $$ Since the original sign bit was 1 (negative) and the magnitude found was 1, the final value is -1.\nTherefore, no matter the size of the variable (8, 16, 32, or 64 bits): if all the bits are 1 (or all the hexadecimal digits are F), the number will always be -1 in Two\u0026rsquo;s Complement.\nIn Practice: Visualizing in Python # If you try to check what we learned here using the Python console, you will get a surprise. Python handles integers differently from languages like C or Java: it has arbitrary precision. This means numbers can have as many bits as needed, with no fixed limit (such as 8 or 32 bits).\nBecause of this, the built-in bin() function does not show Two\u0026rsquo;s Complement by default. It just puts a minus sign in front:\nx = -5 print(bin(x)) # -0b101 # This is not Two\u0026#39;s Complement! It is just \u0026#34;negative 5\u0026#34;. To see how the bits are really stored (like 1111...1011), we need to force Python to limit the number to a given number of bits, using a Bitmask with the AND operator (\u0026amp;).\nIf we want to see the 8-bit representation, we use the mask 0xFF (which is 11111111):\nx = -5 print(bin(x \u0026amp; 0xFF)) # 0b11111011 # Now we are talking! See the bits turned on indicating the negative. If we want to see it in 32 bits, we use 0xFFFFFFFF:\nx = -5 print(bin(x \u0026amp; 0xFFFFFFFF)) # 0b11111111111111111111111111111011 This happens because the \u0026amp; operator treats the number as a pure sequence of bits, revealing the Two\u0026rsquo;s Complement structure that lies \u0026ldquo;under the hood\u0026rdquo;.\nSummary Table # Concept Rule Example from the article Radix complement in base \\(b\\) \\(b^d - N\\), in a system of \\(d\\) digits \\(10^3 - 734 = 266\\) Two\u0026rsquo;s complement \\(2^d - N\\), in a system of \\(d\\) bits \\(2^4 - 5 = 1011_2\\) Arithmetic shortcut flip all the bits and add 1 0101 \\(\\to\\) 1010 \\(\\to\\) 1011 Visual shortcut copy from the right up to the first 1, flip the rest 011100 \\(\\to\\) 100100 Reading a negative subtract \\(2^d\\) from the unsigned value \\(251 - 256 = -5\\) Reading a negative (2nd way) give the MSB weight \\(-2^{d-1}\\) \\(-128 + 64 + 32 + 16 + 8 + 2 + 1 = -5\\) Subtracting by adding \\(A - B = A + \\text{comp}_2(B)\\), discarding the final carry \\(25 - 19 = 6\\) Unsigned range \\(0\\) to \\(2^N - 1\\) 8 bits: 0 to 255 Signed range \\(-2^{N-1}\\) to \\(2^{N-1} - 1\\) 8 bits: \\(-128\\) to \\(+127\\) Why the range is asymmetric there is only one zero, so one negative combination is left over \\(-128\\) has no opposite Overflow (human rule) positive + positive giving negative, or the reverse \\(88 + 46 = -122\\) Overflow (technical rule) the carry into the sign bit differs from the one out of it \\(-76 + (-68) = +112\\) Negative in hexadecimal first digit \\(\\ge 8\\) 0xF0000000 is negative The case of \\(-1\\) all digits equal to F 0xFFFFFFFF is worth \\(-1\\) In Python integers have arbitrary precision; use a mask to see the bits bin(-5 \u0026amp; 0xFF) \\(\\to\\) '0b11111011' Conclusion # The Two\u0026rsquo;s Complement system is an elegant solution: thanks to it, processors can perform additions and subtractions using the same logic circuit (the adder), just flipping bits when necessary. This simplifies chip design and reduces costs.\nWith that, we close the cycle of integers. We know how to represent them, convert them, and operate on them, including the negatives.\nBut what happens when we need precision beyond the integer? How does the computer deal with \\(0.5\\) or \\(3.14\\)? And why does the \u0026amp; operator work as shown above?\nThese questions will be answered in the next articles of the series From Zero to Float. Stay tuned!\n","date":"2026-03-27","externalUrl":null,"permalink":"/2026/03/twos-complement-overflow/","section":"Blog","summary":"To subtract, the processor adds the first number to the complement of the second, so a single adder does both operations. We will see two’s complement, why 8 signed bits range from -128 to +127, and how overflow took down the Ariane 5 and froze Pac-Man on level 256.","title":"How the Computer Subtracts by Adding: Two's Complement and Overflow","type":"posts"},{"content":"","date":"2026-03-27","externalUrl":null,"permalink":"/tags/overflow/","section":"tags","summary":"","title":"overflow","type":"tags"},{"content":" In the previous article, we learned how to read and write numbers in different bases, such as binary and hexadecimal. Now we are going to do arithmetic with those numbers, because computers store data and also process it.\nThe good news is that you do not need to relearn math. The addition and subtraction rules you learned in elementary school for the decimal base (base 10) are universal. The idea of a \u0026ldquo;carry\u0026rdquo; or of \u0026ldquo;borrowing\u0026rdquo; works exactly the same way, whether you are counting apples (decimal) or electrical pulses (binary).\nIn this article, we will revisit these fundamental operations in base 10 to isolate the algorithm and then apply exactly the same logic to perform calculations like a computer.\nDecimal base # The decimal base, also known as base 10, is the most widely used number system in everyday life. It is made up of the digits 0 to 9 and uses a positional system, where the value of each digit depends on its position within the number.\nAddition in the decimal base - practical rule # Adding decimal numbers follows a positional process, where the digits are added column by column, starting from the units (right) and moving to the left.\nThe \u0026ldquo;secret\u0026rdquo; here is to generalize the rule: the carry occurs whenever the sum reaches or exceeds the value of the base. In the decimal case, if it came to 10 or more, carry one.\nExample: Sum \\(67 + 84\\) in base 10 $$ \\begin{array}{r r r} \u0026 {\\scriptstyle 1} \u0026 \\\\ \u0026 6 \u0026 7\\\\ +\u0026 8 \u0026 4\\\\ \\hline 1\u0026 5 \u0026 1 \\end{array} $$Step by step:\nWe add the units column: \\(7 + 4 = 11\\). Since the result has two digits (it went past the base), we put the 1 in the units position and carry the other 1 to the next column (called the \u0026ldquo;carry\u0026rdquo;). We add the tens column: \\(6 + 8 = 14\\) and add the \u0026ldquo;carry\u0026rdquo;, giving \\(14 + 1 = 15\\). Since there are no more columns, 15 is written as the final result. Therefore, the result of the sum is 151.\nThe Carry Movement # The carry occurs when the sum of two digits results in a value greater than 9, since 10 is the base of the decimal system and the carry occurs whenever the sum reaches or exceeds the base, as we have already seen. In that case, we keep the units digit in the corresponding column and carry the tens digit to the next column.\nExample: Sum \\(489 + 276\\) in base 10 $$ \\begin{array}{r r r} {\\scriptstyle 1} \u0026 {\\scriptstyle 1} \u0026\\\\ 4 \u0026 8 \u0026 9\\\\ +2 \u0026 7 \u0026 6\\\\ \\hline 7 \u0026 6 \u0026 5 \\end{array} $$ Units: \\(9 + 6 = 15\\), we write the 5 and carry 1 to the tens column. Tens: \\(8 + 7 = 15\\), we add the \u0026ldquo;carry\u0026rdquo; (\\(15 + 1 = 16\\)), write 6 and carry 1 to the next column. Hundreds: \\(4 + 2 = 6\\), we add the \u0026ldquo;carry\u0026rdquo; (\\(6 + 1 = 7\\)), giving 7. The final result is 765.\nAddition in the decimal base - formal algorithm # Adding decimal numbers follows a positional process, as we saw. To better understand how this will work in other bases, let us look at the formal algorithm: we add the column and check the result.\nIf the value is greater than or equal to the base (in this case, 10), we cannot represent it with a single digit. So we subtract the value of the base (10 in this case) to find what stays in the current column and, in exchange, we send 1 to the next column (the carry).\nLet us repeat the previous example, but now detailing the algorithm formally.\nExample: Sum \\(67 + 84\\) in base 10 (Formal Algorithm) $$ \\begin{array}{r r r l} \u0026 {\\scriptstyle 1} \u0026 \u0026 \\\\ \u0026 6 \u0026 7 \u0026 \\\\ +\u0026 8 \u0026 4 \u0026 \\\\ \\hline \u0026 (15) \u0026 (11) \u0026 \\text{partial sums} \\\\ -\u0026 10 \u0026 10 \u0026 \\text{subtracting the base (10)} \\\\ \\hline 1\u0026 5 \u0026 1 \u0026 \\end{array} $$Step by step through the algorithm:\nUnits: \\(7 + 4 = 11\\). Since \\(11 \\ge 10\\), we compute \\(11 - 10 = 1\\). The 1 stays and we generate a \u0026ldquo;carry\u0026rdquo;. Tens: \\(6 + 8 = 14\\). We add the \u0026ldquo;carry\u0026rdquo; (\\(15\\)). Since \\(15 \\ge 10\\), we compute \\(15 - 10 = 5\\). The 5 stays and we generate a \u0026ldquo;carry\u0026rdquo; for the next column (hundreds). Subtraction in the Decimal Base - Practical Rule # Subtraction also follows a positional process. In this article, we will adopt the equal additions method (or compensation method, which some European schools call the Austrian method), which we will call \u0026ldquo;carry sub\u0026rdquo;.\nWhenever you need to \u0026ldquo;borrow\u0026rdquo;, instead of decreasing the top value, we add 1 to the immediately higher place of the subtrahend (the bottom number). This keeps the difference correct but leaves the look of the calculation cleaner.\nNote on methods Perhaps you learned in school the method of \u0026ldquo;borrowing\u0026rdquo; by crossing out the top number (e.g., the 7 becomes 6 to lend 1 to the 5). Both methods are mathematically equivalent. We will use the compensation method here because it makes the visual organization easier in computational algorithms. It has its own history and a direct connection to the next article in this series, which deals with the method of complements, the idea that hardware uses to avoid having to subtract.\nExample: Subtraction \\(75 - 38\\) (carry sub) $$ \\begin{array}{r r r} \u00267 \u0026 {\\scriptstyle 1}5\\\\ -\u0026{\\scriptstyle 1+}3 \u0026 8\\\\ \\hline \u00263 \u0026 7 \\end{array} $$Step by step:\nUnits: \\(5 - 8\\) is not possible. To get around this, we add 10 to the minuend in the units place (the \\(5\\) becomes 15) and add 1 to the tens place of the subtrahend (the \\(3\\) becomes 4). This way, the difference remains unchanged. Now we subtract the units: \\(15 - 8 = 7\\). Tens: after the adjustment (the 3 became 4), we subtract \\(7 - 4 = 3\\). In the end, we get \\(37\\) as the result.\nThe Borrowing Process - carry sub # In the \u0026ldquo;carry sub\u0026rdquo; method, whenever the upper place is not enough for the subtraction, we add 10 (the value of the base) to the minuend digit in that place and, simultaneously, add 1 to the immediately next place of the subtrahend.\nExample: Subtraction \\(402 - 189\\) (carry sub) $$ \\begin{array}{r r r r} \u00264 \u0026 {\\scriptstyle 1}0 \u0026 {\\scriptstyle 1}2\\\\ -\u0026{\\scriptstyle 1+}1 \u0026 {\\scriptstyle 1+}8 \u0026 9\\\\ \\hline \u00262 \u0026 1 \u0026 3 \\end{array} $$ Units: we need to compute \\(2 - 9\\). Since \\(2 \u003c 9\\), we add 10 to the 2 (it becomes 12) and add 1 to the tens place of the subtrahend (8 becomes 9). Now we subtract: \\(12 - 9 = 3\\). Tens: we now have \\(0 - 9\\). Again it is not possible without \u0026ldquo;borrowing\u0026rdquo;: we add 10 to the 0 (it becomes 10) and add 1 to the hundreds place of the subtrahend (1 becomes 2). Then we compute \\(10 - 9 = 1\\). Hundreds: what is left is \\(4 - 2\\), which is \\(2\\). This way, we obtain the result \\(213\\).\nSubtraction in the Decimal Base - Formal Algorithm # Subtraction also follows a positional process. In the compensation method (carry sub), the algorithmic logic is clear: if the top digit is smaller than the bottom one, we add the value of the base (10 in this case) to the top digit to make the subtraction possible.\nTo keep the equation balanced, we must compensate by adding 1 to the next column of the bottom number (the subtrahend).\nExample: Subtraction \\(75 - 38\\) in base 10 (Formal Algorithm) $$ \\begin{array}{r r r} \u0026 7 \u0026 5^{\\scriptscriptstyle+10} \\\\ -\u0026 3^{\\scriptscriptstyle+1} \u0026 8 \\\\ \\hline \u0026 3 \u0026 7 \\end{array} $$Step by step through the algorithm:\nUnits: \\(5 - 8\\) is not possible. We add the base to the 5 (\\(5 + 10 = 15\\)). Now we compute \\(15 - 8 = 7\\). We compensate by adding \\(+1\\) to the neighbor below (the 3). Tens: Now we have \\(7\\) on top and \\(3 + 1 = 4\\) on the bottom. We compute \\(7 - 4 = 3\\). Other bases # Addition in different bases # Addition in other bases follows the same principle as the decimal base: the \u0026ldquo;carry\u0026rdquo; occurs whenever the result of a sum exceeds the largest value allowed in the base. Now the formal algorithm becomes more evident: if the value of the sum is greater than or equal to the base, we subtract the value of the base to find what stays in the current column and send 1 to the next column.\nLet us go to the practical examples:\nExample: Sum in base 9 Base 9 has digits from 0 to 8. Whenever the sum is greater than or equal to the base (9), the \u0026ldquo;carry\u0026rdquo; is added to the next column. $$ \\begin{array}{c c c c} {\\scriptscriptstyle 1} \u0026 \\scriptscriptstyle 1 \u0026 \u0026 {} \\\\ {} \u0026 6 \u0026 7 \u0026 {} \\\\ {} + \u0026 8 \u0026 4 \u0026 {} \\\\ \\hline 1 \u0026 (15) \u0026 (11) \u0026 {} \\\\ {} - \u0026 9 \u0026 9 \u0026 {\\text{ subtracting the base}} \\\\ \\hline 1 \u0026 6 \u0026 2 \u0026 {} \\end{array} $$Step by step:\nUnits: \\(7 + 4 = 11\\). We write 2, the result of \\(11 - 9\\) (base), and carry 1 to the next column. Tens: \\(6 + 8 = 14\\). We add the \u0026ldquo;carry\u0026rdquo; and get \\(14 + 1 = 15\\). The result is 6 and we carry 1 to the next column. Therefore, the result of the sum \\(67 + 84\\) in base 9 is \\(162_9\\).\nExample: Sum in base 16 Base 16 has digits from 0 to F (representing 15). Whenever the sum is greater than or equal to the base (16), the \u0026ldquo;carry\u0026rdquo; is added to the next column. $$ \\begin{array}{c c c c c} {\\scriptscriptstyle 1} \u0026 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 {} \\\\ {} \u0026 F \u0026 1 \u0026 A \u0026 {} \\\\ {} + \u0026 E \u0026 0 \u0026 9 \u0026 {} \\\\ \\hline 1 \u0026 (29) \u0026 2 \u0026 (19) \u0026 {} \\\\ {} - \u0026 16 \u0026 \u0026 16 \u0026 {\\text{ subtracting the base}} \\\\ \\hline 1 \u0026 D \u0026 2 \u0026 3 \u0026 {} \\end{array} $$Result: \\(\\mathtt{F1A} + \\mathtt{E09} = \\mathtt{1D23_{16}}\\).\nBefore looking at the base 2 examples, it is worth keeping in mind the basic binary addition \u0026ldquo;table\u0026rdquo;, which is where most of the confusion happens:\n\\(0 + 0 = 0\\) \\(0 + 1 = 1\\) \\(1 + 0 = 1\\) \\(1 + 1 = 10_2\\) (Write 0 and carry 1) \\(1 + 1 + 1 = 11_2\\) (Write 1 and carry 1) Example: Sum in base 2 Base 2 has digits 0 and 1. Whenever the sum is greater than or equal to the base (2), the \u0026ldquo;carry\u0026rdquo; is added to the next column. $$ \\begin{array}{c c c c c c c} \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 {} \\\\ {} \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 1 \u0026 {} \\\\ {} + \u0026 1 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \u0026 {} \\\\ \\hline \u0026 (3) \u0026 (2) \u0026 (2) \u0026 (3) \u0026 (2) \u0026 {} \\\\ {} - \u0026 2 \u0026 2 \u0026 2 \u0026 2 \u0026 2 \u0026 {\\text{ subtracting the base}} \\\\ \\hline 1 \u0026 1 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \u0026 {} \\end{array} $$Result: \\(10111_2 + 11011_2 = 110010_2\\).\nIn base 2, we saw that \\(1_2 + 1_2 = 10_2\\) and \\(1_2 + 1_2 + 1_2 = 11_2\\). So it can be easier to add binary numbers if we remember these sums by heart.\nExample: Sum in base 2 using knowledge of the values of 2 and 3 in the base $$ \\begin{array}{c c c c c c c} \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 \u0026 \u0026 {} \\\\ {} \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \u0026 {} \\\\ {} + \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 1 \u0026 {} \\\\ \\hline 1 \u0026 1 \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 {} \\end{array} $$ Three cases that test your understanding # The examples above have, at most, one isolated \u0026ldquo;carry\u0026rdquo; per column. Three situations depart from this pattern and are worth studying, because they are where the calculation tends to go off the rails.\nTwo maximum digits in the same column. In base 8, the largest digit is 7. Adding \\(77_8 + 77_8\\) puts the algorithm in the worst possible case in every column:\n$$ \\begin{array}{c c c c} \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 {} \\\\ {} \u0026 7 \u0026 7 \u0026 {} \\\\ {} + \u0026 7 \u0026 7 \u0026 {} \\\\ \\hline \u0026 (15) \u0026 (14) \u0026 {} \\\\ {} - \u0026 8 \u0026 8 \u0026 {\\text{ subtracting the base}} \\\\ \\hline 1 \u0026 7 \u0026 6 \u0026 {} \\end{array} $$Result: \\(77_8 + 77_8 = 176_8\\). Check it in decimal: \\(77_8 = 63\\), and \\(63 + 63 = 126 = 176_8\\) (\\(1 \\times 64 + 7 \\times 8 + 6\\)).\nCarry chain. In \\(2\\mathtt{FF}_{16} + 1\\mathtt{FE}_{16}\\), the \u0026ldquo;carry\u0026rdquo; does not stay stuck in one column: it is born in the units and crosses the entire calculation.\n$$ \\begin{array}{c c c c c} {} \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \u0026 {} \\\\ {} \u0026 2 \u0026 F \u0026 F \u0026 {} \\\\ {} + \u0026 1 \u0026 F \u0026 E \u0026 {} \\\\ \\hline {} \u0026 4 \u0026 (31) \u0026 (29) \u0026 {} \\\\ {} - \u0026 {} \u0026 16 \u0026 16 \u0026 {\\text{ subtracting the base}} \\\\ \\hline {} \u0026 4 \u0026 F \u0026 D \u0026 {} \\end{array} $$Result: \\(2\\mathtt{FF}_{16} + 1\\mathtt{FE}_{16} = 4\\mathtt{FD}_{16}\\). In decimal: \\(767 + 510 = 1277\\), and \\(4 \\times 256 + 15 \\times 16 + 13 = 1277\\).\nIn binary this chain effect is even clearer, and it is worth seeing it drawn. The figure below follows an extreme case of an 8-bit sum, \\(01111111_2 + 00000001_2\\), that is, \\(127 + 1\\). The \u0026ldquo;carry\u0026rdquo; is born in the rightmost column and has to travel, one by one, through all seven following columns before the result is ready. The three panels at the bottom highlight the three moments of this journey:\nAn extreme case of an 8-bit sum: the carry is born on the right and has to travel through all the columns to the left. This is where the name ripple carry comes from: each column can only finish after the previous one has handed over its carry. On paper this is just work; in hardware, it is time, and that is why modern processors use smarter adders, which compute the carry in parallel instead of waiting for it.\nAdding three numbers at once. Here lies the most beautiful trap of the algorithm. Everyone learns \u0026ldquo;carry one\u0026rdquo;, but the carry is only worth one because we are adding two numbers. With three addends, a column can generate a carry of 2.\n$$ \\begin{array}{c c c c c c c c} {} \u0026 {} \u0026 \\scriptscriptstyle 2 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 \\scriptscriptstyle 1 \u0026 {} \\\\ {} \u0026 {} \u0026 {} \u0026 1 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \\\\ {} \u0026 {} \u0026 {} \u0026 1 \u0026 1 \u0026 1 \u0026 0 \u0026 1 \\\\ {} + \u0026 {} \u0026 {} \u0026 1 \u0026 1 \u0026 0 \u0026 1 \u0026 1 \\\\ \\hline {} \u0026 1 \u0026 0 \u0026 0 \u0026 1 \u0026 1 \u0026 0 \u0026 0 \\end{array} $$Follow the column that overflows: in the fifth position we have \\(1 + 1 + 1 = 3\\), plus the carry 1 that arrived from the previous column, which makes \\(4\\). Since \\(4 = 100_2\\), \\(0\\) stays in the column and 2 go to the next one. Result: \\(10100_2 + 11101_2 + 11011_2 = 1001100_2\\), or \\(20 + 29 + 27 = 76\\).\nThe \u0026ldquo;carry\u0026rdquo; is not always one The general rule is not \u0026ldquo;write the remainder and carry 1\u0026rdquo;, but rather: write the remainder of the division by the base and carry the quotient. With two addends the quotient never exceeds 1, and the simplified rule works. With three or more, it can be 2 or more, and that is exactly where a calculation done on autopilot goes wrong.\nSubtraction in different bases # Subtraction in other bases follows the same principle we saw in the decimal base (carry sub/compensation): if the top digit is smaller than the bottom one, we add the base to the top one and compensate by adding 1 to the neighbor below.\nAgain, the formal algorithm becomes more evident: if the value of the subtraction is negative, we add the value of the base to the top digit to make the subtraction possible and add 1 to the next column of the bottom number (the subtrahend) to keep the equation balanced.\nExample: Subtraction in base 8 In base 8, the digits go from 0 to 7. Whenever the subtraction results in a negative number, we borrow 1 from the next column. $$ \\begin{array}{c c c} {} \u0026 5 \u0026 2^{\\scriptscriptstyle+8} \\\\ {} - \u0026 1^{\\scriptscriptstyle+1} \u0026 7 \\\\ \\hline {} \u0026 3 \u0026 3 \\end{array} $$Step by step:\nUnits: \\(2 - 7\\) is not possible. We add the base (8) to the top digit (the \\(2\\) becomes \\(10\\)) and compensate with \\(+1\\) in the next column of the subtrahend (the \\(1\\) becomes \\(2\\)). Now \\(10 - 7 = 3\\). Next column: \\(5 - 2 = 3\\). Result: \\(52_8 - 17_8 = 33_8\\).\nCheck: \\(52_8 = 42\\) and \\(17_8 = 15\\); \\(42 - 15 = 27\\), which in base 8 is \\(3 \\times 8 + 3 = 33_8\\).\nExample: Subtraction in base 16 In base 16, the digits go from 0 to F. Whenever the subtraction results in a negative number, we borrow 1 from the next column. $$ \\begin{array}{c c c} {} \u0026 5 \u0026 2^{\\scriptscriptstyle+16} \\\\ {} - \u0026 1^{\\scriptscriptstyle+1} \u0026 7 \\\\ \\hline {} \u0026 3 \u0026 B \\end{array} $$Step by step:\nUnits: \\(2 - 7\\) is not possible. We add the base (16) to the top digit (the \\(2\\) becomes \\(18\\)) and compensate with \\(+1\\) in the next column of the subtrahend (the \\(1\\) becomes \\(2\\)). Now \\(18 - 7 = 11\\), which in hexadecimal is the digit \\(\\mathtt{B}\\). Next column: \\(5 - 2 = 3\\). Result: \\(52_{16} - 17_{16} = 3\\mathtt{B}_{16}\\).\nCheck: \\(52_{16} = 82\\) and \\(17_{16} = 23\\); \\(82 - 23 = 59\\), which in base 16 is \\(3 \\times 16 + 11 = 3\\mathtt{B}_{16}\\).\nNotice that the calculation is the same as in the previous two examples: only the borrowed value changes, which is always the base.\nIn base 2, we can use the knowledge already acquired from the \u0026ldquo;table\u0026rdquo; to make subtraction easier.\nExample: Subtraction in base 2 $$ \\begin{array}{c c c c c c c} {} \u0026 1 \u0026 1 \u0026 {\\scriptscriptstyle1}0 \u0026 {\\scriptscriptstyle1}0 \u0026 1 \u0026 {\\scriptscriptstyle1}0 \\\\ {} - \u0026 1 \u0026 0^{\\scriptscriptstyle+1} \u0026 1^{\\scriptscriptstyle+1} \u0026 1 \u0026 0^{\\scriptscriptstyle+1} \u0026 1 \\\\ \\hline \u0026 0 \u0026 0 \u0026 0 \u0026 1 \u0026 0 \u0026 1 \\end{array} $$Step by step, from right to left:\n\\(0 - 1\\) does not work. We add the base (2) to the top, which becomes \\(10_2\\), and compensate with \\(+1\\) in the next bottom column. \\(2 - 1 = 1\\). Top \\(1\\), bottom \\(0 + 1 = 1\\). \\(1 - 1 = 0\\), no compensation. Top \\(0\\), bottom \\(1\\). We add the base: \\(2 - 1 = 1\\), and compensate. Top \\(0\\), bottom \\(1 + 1 = 2\\). We add the base: \\(2 - 2 = 0\\), and compensate. Top \\(1\\), bottom \\(0 + 1 = 1\\). \\(1 - 1 = 0\\). Top \\(1\\), bottom \\(1\\). \\(1 - 1 = 0\\). Result: \\(110010_2 - 101101_2 = 101_2\\).\nCheck: \\(110010_2 = 50\\) and \\(101101_2 = 45\\); \\(50 - 45 = 5\\), which in binary is \\(101_2\\).\nFormal Algorithms and Implementation # For a computer scientist, understanding arithmetic includes knowing how to do the calculation on paper and understanding the algorithm that can be programmed. Below, we present the generalized logic for any Base \\(B\\), first in pseudocode and then in Python.\nAddition Algorithm (Generalized) # The central logic depends on two integer operations: the remainder of the division (mod in the pseudocode, % in Python) determines the digit that stays, and the integer division (\\(\\lfloor \\cdot \\rfloor\\) in the pseudocode, // in Python) determines the \u0026ldquo;carry\u0026rdquo;.\nTo begin with, let us look at pseudocode, which is a way of writing algorithms independently of any programming language. It may make things easier to understand before we see the Python code.\nPseudocode:\nADD(A, B, n, b) % A[1..n] and B[1..n]: the digits of the two addends in base b, right-aligned % and padded with leading zeros; A[n] and B[n] are the units carry := 0 for i = n downto 1 sum := A[i] + B[i] + carry R[i] := sum mod b % the digit that stays carry := ⌊sum / b⌋ % the carry for the next column if carry \u0026gt; 0 R[0] := carry % the last carry becomes an extra digit return R[0..n] else return R[1..n] Python implementation:\nFor the Python code, we will use an auxiliary string to map digits greater than 9: DIGITS = \u0026quot;0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\u0026quot;\nDIGITS = \u0026#34;0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\u0026#34; def add_bases(n1: str, n2: str, base: int) -\u0026gt; str: # Reverse the strings to make iterating from right to left easier n1, n2 = n1[::-1].upper(), n2[::-1].upper() # Equalize the string lengths with zeros on the right (which are on the left in the real number) max_len = max(len(n1), len(n2)) n1 = n1.ljust(max_len, \u0026#39;0\u0026#39;) n2 = n2.ljust(max_len, \u0026#39;0\u0026#39;) carry = 0 result = [] for i in range(max_len): # Convert character to numeric value (e.g., \u0026#39;A\u0026#39; -\u0026gt; 10) v1 = DIGITS.index(n1[i]) v2 = DIGITS.index(n2[i]) partial_sum = v1 + v2 + carry # Base logic kept_digit = partial_sum % base carry = partial_sum // base result.append(DIGITS[kept_digit]) if carry \u0026gt; 0: result.append(DIGITS[carry]) # Reverse back to return the correct string return \u0026#34;\u0026#34;.join(result[::-1]) # --- Execution Examples (Based on the article\u0026#39;s examples) --- # Base 10: 67 + 84 print(f\u0026#34;Base 10: {add_bases(\u0026#39;67\u0026#39;, \u0026#39;84\u0026#39;, 10)}\u0026#34;) # Result: 151 # Base 9: 67 + 84 print(f\u0026#34;Base 9: {add_bases(\u0026#39;67\u0026#39;, \u0026#39;84\u0026#39;, 9)}\u0026#34;) # Result: 162 (Note that 7+4=11, which is 12 in base 9) # Base 16: F1A + E09 print(f\u0026#34;Base 16: {add_bases(\u0026#39;F1A\u0026#39;, \u0026#39;E09\u0026#39;, 16)}\u0026#34;) # Result: 1D23 # Base 2: 10111 + 11011 print(f\u0026#34;Base 2: {add_bases(\u0026#39;10111\u0026#39;, \u0026#39;11011\u0026#39;, 2)}\u0026#34;) # Result: 110010 Subtraction Algorithm (Carry Sub Method) # In this method, when the top digit is smaller, we add the base b to it and mark a compensation (+1) to be added to the bottom digit in the next column.\nPseudocode:\nSUBTRACT(A, B, n, b) % A[1..n]: minuend; B[1..n]: subtrahend, with A ≥ B; both in base b, % right-aligned and padded with leading zeros compensation := 0 for i = n downto 1 bottom := B[i] + compensation top := A[i] if top \u0026lt; bottom top := top + b % \u0026#34;borrows\u0026#34; the base compensation := 1 else compensation := 0 R[i] := top - bottom return R[1..n] % leading zeros do not change the value Python implementation:\nDIGITS = \u0026#34;0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\u0026#34; def subtract_bases(n1: str, n2: str, base: int) -\u0026gt; str: # This algorithm can only produce non-negative results. Without the check # below, subtract_bases(\u0026#39;38\u0026#39;, \u0026#39;75\u0026#39;, 10) would silently return \u0026#39;63\u0026#39;. # Comparing numerals of the same base does not require converting them: just # look at the length (without leading zeros) and, in case of a tie, compare # digit by digit (the symbols in DIGITS are already in increasing order). a = n1.upper().lstrip(\u0026#39;0\u0026#39;) or \u0026#39;0\u0026#39; b = n2.upper().lstrip(\u0026#39;0\u0026#39;) or \u0026#39;0\u0026#39; if (len(a), a) \u0026lt; (len(b), b): raise ValueError(f\u0026#34;{n1} \u0026lt; {n2} in base {base}: result would be negative\u0026#34;) n1, n2 = n1[::-1].upper(), n2[::-1].upper() max_len = max(len(n1), len(n2)) n1 = n1.ljust(max_len, \u0026#39;0\u0026#39;) n2 = n2.ljust(max_len, \u0026#39;0\u0026#39;) compensation = 0 result = [] for i in range(max_len): v_top = DIGITS.index(n1[i]) v_bottom = DIGITS.index(n2[i]) + compensation if v_top \u0026lt; v_bottom: # Apply the Carry Sub logic (add the base on top) v_top += base compensation = 1 else: compensation = 0 difference = v_top - v_bottom result.append(DIGITS[difference]) # Remove resulting leading zeros (e.g., 037 -\u0026gt; 37) return \u0026#34;\u0026#34;.join(result[::-1]).lstrip(\u0026#39;0\u0026#39;) or \u0026#34;0\u0026#34; # --- Execution Examples --- # Base 10: 75 - 38 print(f\u0026#34;Base 10: {subtract_bases(\u0026#39;75\u0026#39;, \u0026#39;38\u0026#39;, 10)}\u0026#34;) # Result: 37 # Base 8: 52 - 17 (Article example: we borrow 8) print(f\u0026#34;Base 8: {subtract_bases(\u0026#39;52\u0026#39;, \u0026#39;17\u0026#39;, 8)}\u0026#34;) # Result: 33 (Since 2 became 10 (2+8), 10-7=3. The 1 became 2, 5-2=3) # Base 16: 52 - 17 print(f\u0026#34;Base 16: {subtract_bases(\u0026#39;52\u0026#39;, \u0026#39;17\u0026#39;, 16)}\u0026#34;) # Result: 3B (Since 2 became 18 (2+16), 18-7=11(B). The 1 became 2, 5-2=3) # Base 2: 1101010 - 101101 (Adapted example) print(f\u0026#34;Base 2: {subtract_bases(\u0026#39;1101010\u0026#39;, \u0026#39;101101\u0026#39;, 2)}\u0026#34;) # Result: 111101 Summary Table # Situation Rule Example from the article Addition, column by column write \\(s \\bmod b\\) and carry \\(s \\div b\\) \\(67_9 + 84_9 = 162_9\\) Carry with two addends the carry never exceeds 1 \\(1_2 + 1_2 = 10_2\\) Carry with three addends the carry can be 2 or more \\(10100_2 + 11101_2 + 11011_2 = 1001100_2\\) Worst case of addition two maximum digits in the same column \\(77_8 + 77_8 = 176_8\\) Carry chain the carry crosses several columns \\(2\\mathtt{FF}_{16} + 1\\mathtt{FE}_{16} = 4\\mathtt{FD}_{16}\\) Subtraction (carry sub) if the top is smaller, add the base on top and \\(+1\\) in the next bottom column \\(52_8 - 17_8 = 33_8\\) How much is borrowed always the value of the base 8 in base 8, 16 in base 16, 2 in base 2 Check convert to decimal and redo the calculation \\(42 - 15 = 27 = 33_8\\) Addition in code kept_digit = partial_sum % base, carry = partial_sum // base add_bases('F1A','E09',16) Subtraction in code if v_top \u0026lt; v_bottom: v_top += base subtract_bases('52','17',16) Limit of the algorithm only non-negative results subtract_bases('38','75',10) raises ValueError Conclusion and Next Steps # As we have seen, positional arithmetic is consistent: it does not matter whether we have 2 symbols (binary) or 16 (hexadecimal), the algorithm is the same. The only thing that changes is the value that triggers the \u0026ldquo;carry\u0026rdquo; or the \u0026ldquo;borrow\u0026rdquo;.\nMastering these operations on paper is excellent for understanding the logic, but it raises an important question: does computer hardware really perform subtractions by borrowing neighboring bits?\nIn practice, no. A full subtractor has a cost similar to that of a full adder, because both are made essentially of the same logic gates; the problem is the waste.\nIt is worth taking a close look at the circuit we are talking about. The full adder is the piece that carries out, in silicon, exactly one column of the calculation we did on paper: it receives two bits and the carry that arrived from the previous column, and returns the sum bit and the carry for the next column. It is five logic gates, that is all:\nA 1-bit full adder: two XOR gates produce the sum bit, and two AND gates plus an OR decide whether there is a carry. Wiring eight of these pieces in a row, with the carry of one feeding into the next, gives exactly the 8-bit sum we drew above, including the chain wait. And a full subtractor is of similar size: the same gates plus two inverters, wired differently. That is where the waste lies.\nKeeping two separate circuits, one for adding and one for subtracting, uses up processor area doing almost identical work twice and, worse, forces the design to decide which of the two to use for each operation.\nThe way out in modern computing is elegant: turn every subtraction into an addition, so that a single adder handles both operations. This is the concept of Two\u0026rsquo;s Complement and of signed numbers, and it is exactly this representation of negative numbers that we will talk about in the next article.\nSee you then!\n","date":"2026-03-25","externalUrl":null,"permalink":"/2026/03/arithmetic-operations-number-bases/","section":"Blog","summary":"The addition and subtraction rules of base 10 hold in any base: only the value that triggers the carry changes. We do arithmetic in binary, octal, and hexadecimal, see why with three addends the carry can be worth 2, and implement both algorithms in Python.","title":"Computer Arithmetic: Addition and Subtraction in Binary, Octal, and Hexadecimal","type":"posts"},{"content":"","date":"2026-03-25","externalUrl":null,"permalink":"/tags/decimal/","section":"tags","summary":"","title":"decimal","type":"tags"},{"content":"","date":"2026-03-25","externalUrl":null,"permalink":"/tags/octal/","section":"tags","summary":"","title":"octal","type":"tags"},{"content":"","date":"2026-03-23","externalUrl":null,"permalink":"/tags/base-conversion/","section":"tags","summary":"","title":"base conversion","type":"tags"},{"content":"","date":"2026-03-23","externalUrl":null,"permalink":"/tags/conversao-de-bases/","section":"tags","summary":"","title":"conversão de bases","type":"tags"},{"content":" Computer systems do not \u0026ldquo;understand\u0026rdquo; numbers the way we do. In hardware, all information (instructions, addresses, characters, and numeric values) must be encoded as electrical signals with a few stable states. That is why the conceptual foundation of digital computing is the use of number systems, especially those that fit naturally with electronics and with the organization of bits.\nIn practice, you will switch between representations depending on the context: binary is the circuit\u0026rsquo;s native language; hexadecimal (and, historically, octal) exist to make bit patterns readable; and decimal remains the most convenient system for human interaction, reading, and checking results.\nIn this article, you will learn:\nwhat positional notation is and how it works; how to convert integers between common bases (decimal, binary, octal, and hexadecimal); when you can speed up conversions by grouping/expanding digits (bases related by a power). The focus here is on nonnegative integers. In future articles, we will cover signed conversions (including two\u0026rsquo;s complement representation) and floating-point numbers.\nA few quick definitions before we start:\nBinary (base-2) is the base of digital systems: bits and logical operations work directly with 0 and 1. Hexadecimal (base-16) and Octal (base-8) are used to compact and \u0026ldquo;see\u0026rdquo; binary sequences without losing the correspondence with the bits. The Decimal system (base-10) is the everyday standard and appears naturally in inputs/outputs, documentation, and manual checking. Other systems almost never appear in computing, but they left a legacy in everyday life: Duodecimal (base-12) survives in counting by dozens and grosses, in the 12 hours on a clock face, and in the 12 inches of a foot; and Sexagesimal (base-60), inherited from the Babylonians, is the reason a minute has 60 seconds and a degree has 60 arc minutes. Types of Number Systems # A number system is defined by its base (radix), which determines how many symbols exist and what the weights of each position are. Below are the four systems most common in computing.\nBefore listing them, a contrast makes the word \u0026ldquo;positional\u0026rdquo; much more concrete. All the systems in this article are weighted positional: the same symbol is worth different things depending on the position it occupies. Not every number system works this way.\nSystem How it writes 1989 Value of the highlighted symbol Decimal (positional) 1989 900, because it is in the hundreds place Roman (non-positional) MCMLXXXIX 50, anywhere in the sequence In Roman numerals, L is worth 50 wherever it is, and what changes the result is the relative order of the symbols: in IX the I subtracts, in XI it adds. In decimal, the same 9 is worth 900 in one position and 9 in another: the position carries the weight. This difference is exactly what makes positional arithmetic mechanical: adding two numbers becomes a column procedure, with carries, that works the same for 12 or for 12 million.\nWhere the notation we use comes from The positional system of ten symbols with an explicit zero was born in India between the 5th and 7th centuries, was systematized by mathematicians such as Brahmagupta (7th century, who treated zero as a number and not just as an empty space), and reached Europe through the Arab world, hence the name Hindu-Arabic numeral system. Its popularization in the West is usually credited to the Liber Abaci (1202), by Leonardo of Pisa, known as Fibonacci. What made it superior to the Roman system was the positional zero: without it you cannot tell \\(15\\) from \\(105\\) without inventing a blank space, and without that distinction there is no column algorithm.\nDecimal Number System # The Decimal system is a base-10 number system. It uses ten digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. The positional value of each digit is a power of 10 (for example, \\(10^{0}\\), \\(10^{1}\\), \\(10^{2}\\)). It is the standard system for everyday counting and calculation. Binary Number System # The Binary system is a base-2 number system. It uses two digits: 0 and 1. The positional value of each digit is a power of 2 (for example, \\(2^{0}\\), \\(2^{1}\\), \\(2^{2}\\)). The Binary system is the basis for data representation in computers and digital electronics. Octal Number System # The Octal system is a base-8 number system. It uses eight digits: 0, 1, 2, 3, 4, 5, 6, and 7. The positional value of each digit is a power of 8 (for example, \\(8^{0}\\), \\(8^{1}\\), \\(8^{2}\\)). It is often used to simplify the representation of binary numbers by grouping them into sets of three bits. Hexadecimal Number System # The Hexadecimal system is a base-16 number system. It uses sixteen symbols: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, and F. The symbols A to F represent decimal values 10 to 15 (for example, \\(A = 10\\), \\(B = 11\\), …, \\(F = 15\\)). The positional value of each digit is a power of 16 (for example, \\(16^{0}\\), \\(16^{1}\\), \\(16^{2}\\)). Hexadecimal simplifies binary by representing every 4 bits as one digit (from 0 to F). Using the letters A to F was neither obvious nor an immediate consensus: there were competing notations until the current standard settled (Hexadecimal). In summary:\nSystem (base) Available symbols Positional value (weights) Typical use in computing Quick conversion with binary (when applicable) Decimal (10) 0 to 9 powers of 10: \\(10^0,10^1,10^2,\\dots\\) Human interaction, reading, documentation, manual checking of results not applicable Binary (2) 0 and 1 powers of 2: \\(2^0,2^1,2^2,\\dots\\) Native hardware representation (bits), logical operations not applicable Octal (8) 0 to 7 powers of 8: \\(8^0,8^1,8^2,\\dots\\) Readability/compaction of bits (historical context and some systems) \\(8=2^3\\): 1 octal digit ↔ 3 bits Hexadecimal (16) 0 to 9 and A to F (\\(A=10,\\dots,F=15\\)) powers of 16: \\(16^0,16^1,16^2,\\dots\\) Addresses, memory dumps, readable bit patterns (very common) \\(16=2^4\\): 1 hex digit ↔ 4 bits Let us look more closely at what we mean by positional value.\nIntegers and Positional Notation # As highlighted earlier, this article focuses on integers and their representations in different bases. The positional notation is the most common method for representing numbers in any base: the value of each digit depends on its position, and each position has a weight given by a power of the base.\nIn practical terms, this idea explains why:\nthe positional expansion lets us convert a number from any base to decimal (by adding up weights); successive divisions \u0026ldquo;extract\u0026rdquo; the digits of the number in the desired base, from least significant to most significant. A number \\(N\\) in a base \\(b\\) can be represented as:\n$$ N_b = \\sum_{i=0}^{n} d_i \\cdot b^i = d_n \\cdot b^n + d_{n-1} \\cdot b^{n-1} + ... + d_1 \\cdot b^1 + d_0 \\cdot b^0 $$where \\(d_i\\) are the digits of the number and \\(b\\) is the base. Each digit \\(d_i\\) must satisfy \\(0 \\leq d_i \u003c b\\).\nNote a consequence of this restriction: since the base is never among the available symbols, the value of the base itself is always written 10, in any base. In binary, \\(10_2 = 2\\); in octal, \\(10_8 = 8\\); in hexadecimal, \\(10_{16} = 16\\). In other words, \u0026ldquo;10\u0026rdquo; does not mean \u0026ldquo;ten\u0026rdquo;: it means \u0026ldquo;one times the base, plus zero units\u0026rdquo;.\nNote: the same logic holds for the fractional part, using negative powers (\\(b^{-1}, b^{-2}, ...\\)). Since the focus here is on integers, we will come back to fractions and floating point in dedicated articles.\nFor example, the number \\(123_{10}\\) in decimal base can be expanded as:\n$$ 123_{10} = 1 \\cdot 10^2 + 2 \\cdot 10^1 + 3 \\cdot 10^0 = 100 + 20 + 3 $$From here on, we will use the following notations for bases in this article:\n\\(s\\) as the source base (s for source); \\(r\\) as the target base (r for result); \\(b\\) only when the base is generic in a formula. Conversion Methods # To convert integers between bases, you do not need to memorize dozens of rules. In fact, we only need to master two fundamental methods, using the Decimal system as a \u0026ldquo;bridge\u0026rdquo; or \u0026ldquo;universal translator\u0026rdquo;:\nPositional Expansion (Multiplication): Used when we want to find out how much a number in any base is worth in Decimal. Successive Divisions: Used when we have a Decimal value and want to encode it into another base. From Any Base to Decimal (Expansion) # When you have a number in binary, octal, or hexadecimal and want to know its numeric value (\u0026ldquo;what is this worth in my everyday life?\u0026rdquo;), you use Positional Expansion.\nThe process is simple: add up the value of each digit multiplied by the weight of its position (the base raised to the power of the position).\n$$ N_{10} = \\sum (\\text{digit} \\times \\text{base}^{\\text{position}}) $$ Example: Convert \\(1101_2\\) to Decimal We identify the base (\\(2\\)) and the positions (starting from 0, right to left). $$ \\begin{align*} 1101_2 \u0026 = 1 \\cdot 2^3 + 1 \\cdot 2^2 + 0 \\cdot 2^1 + 1 \\cdot 2^0 \\\\ \u0026 = 1 \\cdot 8 + 1 \\cdot 4 + 0 \\cdot 2 + 1 \\cdot 1 \\\\ \u0026 = 8 + 4 + 0 + 1 \\\\ \u0026 = 13_{10} \\end{align*} $$Verification tip: in binary, \\(1101_2\\) must lie between \\(8_{10}\\) (represented as \\(1000_2\\)) and \\(16_{10}\\) (represented as \\(10000_2\\)), so \\(13_{10}\\) makes sense.\nThis same computation fits in a figure, and it is worth looking at it even if the expansion above is already clear, because they are two paths to the same idea. The ruler below puts the weight of each position right above the corresponding bit and, below it, the term that bit contributes. The bit that is off appears in gray precisely because it adds nothing:\nThe four bits of 1101 in binary with the weight of each position and the corresponding term. The bit that is off, in gray, contributes zero. Example: Convert \\(2F_{16}\\) to Decimal Remember that in Hexadecimal, \\(F = 15\\). $$ \\begin{align*} 2F_{16} \u0026 = 2 \\cdot 16^1 + 15 \\cdot 16^0 \\\\ \u0026 = 32 + 15 \\\\ \u0026 = 47_{10} \\end{align*} $$ From Decimal to Any Base (Successive Divisions) # When you have a decimal number and need to represent it in binary, octal, or any other base, the standard method is Successive Divisions.\nThe algorithm is:\n1 Divide by the base Divide the decimal number by the target base. 2 Write down the remainder Write down the remainder (it will be one of the digits). 3 Divide the quotient Take the quotient and divide it by the base again. 4 Repeat until zero Repeat until the quotient is zero. 5 Read the remainders from bottom to top The result is formed by the remainders read from last to first (bottom to top). Example: Convert \\(45_{10}\\) to Binary (Base 2) $$ \\begin{align*} 45 \\div 2 \u0026 = 22 \\quad \u0026 \\text{remainder } \\mathbf{1} \\\\ 22 \\div 2 \u0026 = 11 \\quad \u0026 \\text{remainder } \\mathbf{0} \\\\ 11 \\div 2 \u0026 = 5 \\quad \u0026 \\text{remainder } \\mathbf{1} \\\\ 5 \\div 2 \u0026 = 2 \\quad \u0026 \\text{remainder } \\mathbf{1} \\\\ 2 \\div 2 \u0026 = 1 \\quad \u0026 \\text{remainder } \\mathbf{0} \\\\ 1 \\div 2 \u0026 = 0 \\quad \u0026 \\text{remainder } \\mathbf{1} \\end{align*} $$Reading from bottom to top: \\(\\mathbf{101101_2}\\).\nVerification tip: convert \\(101101_2\\) back to decimal via positional expansion and check that you get \\(45_{10}\\) again.\nThe \u0026ldquo;Decimal Bridge\u0026rdquo; Method (Base \\(A \\to\\) Base \\(B\\)) # What if we want to convert from Base 9 to Base 4? Or from Base 5 to Base 7?\nSince these bases do not talk directly to each other, we use Decimal as an intermediary. The process has two steps:\nSource \\(\\to\\) Decimal (using Expansion). Decimal \\(\\to\\) Target (using Divisions). Example: Convert \\(87_9\\) to Base 4 Step 1: Base 9 to Decimal $$ 87_9 = 8 \\cdot 9^1 + 7 \\cdot 9^0 = 72 + 7 = \\mathbf{79_{10}} $$Step 2: Decimal (79) to Base 4 $$ \\begin{align*} 79 \\div 4 \u0026 = 19 \\quad \u0026 \\text{remainder } \\mathbf{3} \\\\ 19 \\div 4 \u0026 = 4 \\quad \u0026 \\text{remainder } \\mathbf{3} \\\\ 4 \\div 4 \u0026 = 1 \\quad \u0026 \\text{remainder } \\mathbf{0} \\\\ 1 \\div 4 \u0026 = 0 \\quad \u0026 \\text{remainder } \\mathbf{1} \\end{align*} $$Reading the remainders from bottom to top: \\(\\mathbf{1033_4}\\).\nTherefore, \\(87_9 = 1033_4\\).\nOne more example:\nExample: Convert \\(2F_{16}\\) to base \\(2\\) Convert from hexadecimal to decimal. Note that \\(F\\) is a symbol, not a number. Before doing any computation, replace each symbol with the value it represents (here, \\(F = 15\\)): $$ 2F_{16} = 2 \\times 16^1 + F \\times 16^0 = 2 \\times 16^1 + 15 \\times 16^0 $$ Now, with everything in decimal: $$ 2 \\times 16 + 15 \\times 1 = 32 + 15 = 47_{10} $$ Convert from decimal to binary: $$ \\begin{align*} 47 \\div 2 \u0026 = 23 \u0026 \\quad \\text{remainder } 1 \\\\ 23 \\div 2 \u0026 = 11 \u0026 \\quad \\text{remainder } 1 \\\\ 11 \\div 2 \u0026 = 5 \u0026 \\quad \\text{remainder } 1 \\\\ 5 \\div 2 \u0026 = 2 \u0026 \\quad \\text{remainder } 1 \\\\ 2 \\div 2 \u0026 = 1 \u0026 \\quad \\text{remainder } 0 \\\\ 1 \\div 2 \u0026 = 0 \u0026 \\quad \\text{remainder } 1 \\end{align*} $$ Reading from bottom to top: \\(2F_{16} = 101111_2\\).\nVerification tip: you can check the result in hexadecimal by grouping \\(101111_2\\) into 4 bits with padding (filling with zeros) on the left, which gives \\(0010\\) and \\(1111\\), that is, \\(2F_{16}\\), which is where we started.\nExample: Convert \\(1101_2\\) to hexadecimal (\\(16\\)) Convert to decimal: $$ 1101_2 = 1 \\times 2^3 + 1 \\times 2^2 + 0 \\times 2^1 + 1 \\times 2^0 = 13_{10} $$ Convert decimal to hexadecimal: $$ 13_{10} = D_{16} $$ Thus, \\(1101_2 = D_{16}\\).\nVerification tip: since 4 bits form 1 hexadecimal digit, \\(1101_2\\) must indeed map to \\(D_{16}\\).\nThese last two cases in particular can be simplified, without going through decimal, because \\(16 = 2^4\\). Thus, each hexadecimal digit corresponds exactly to 4 binary bits. See the section Making Conversion Easier When the Bases Are Related by a Power for more details. But the \u0026ldquo;via decimal\u0026rdquo; approach works for any pair of bases.\nGeneral Summary of the Methods # Goal Method Find the value (Any \\(\\to\\) Decimal) Positional Expansion (Sum of the products) Encode the value (Decimal \\(\\to\\) Any) Successive Divisions (Read remainders bottom to top) Change of representation (Base \\(A \\to\\) Base \\(B\\)) Bridge via Decimal (Expansion \\(\\to\\) Division) Common Mistakes (and How to Avoid Them) # Reading the remainders in the wrong order: in successive divisions, the result is read from bottom to top. Forgetting the base in the subscript: always keep the notation (\\(N_s\\) and \\(N_r\\)) so you do not mix up interpretations. When grouping (binary→octal/hex), group starting from the right; if the leftmost group is short of bits, fill it with zeros (padding). In the \u0026ldquo;via decimal\u0026rdquo; method, forgetting to convert the symbols \\(A\\) to \\(F\\) into the values \\(10\\) to \\(15\\). Making Conversion Easier When the Bases Are Related by a Power # If \\(s\\) and \\(r\\) have a power relationship (\\(s = r^k\\) or \\(r = s^k\\)), you can convert directly by grouping or expanding digits, without long computations.\nThe most common practical rule in computing is:\nif you are going from binary to a larger base, group bits; if you are going from a larger base to binary, expand each digit into bits. Case 1: Smaller Base to Larger Base (\\(s \u003c r\\)) # If \\(r = s^k\\), each digit in base \\(r\\) corresponds exactly to \\(k\\) digits in base \\(s\\). In computing, the classic case is \\(s=2\\) and \\(r \\in \\{8,16\\}\\), where:\n\\(8 = 2^3\\) (so \\(k=3\\)); \\(16 = 2^4\\) (so \\(k=4\\)). Example: Conversion from Binary (\\(2\\)) to Octal (\\(8\\)) We know that \\(8 = 2^3\\), so each group of 3 bits represents one octal digit.\nStep 1: Split the binary number into blocks of 3 bits, from right to left $$ 101011_2 $$We group: $$ \\mathbf{101} \\quad \\mathbf{011} $$Step 2: Convert each group to octal $$ 101_2 = 5_8, \\quad 011_2 = 3_8 $$Final result: $$ 101011_2 = 53_8 $$ Example: Conversion from Binary (\\(2\\)) to Hexadecimal (\\(16\\)) Since \\(16 = 2^4\\), we group the digits in fours.\nStep 1: Split the binary number into blocks of 4 bits $$ 11010101_2 $$We group: $$ \\mathbf{1101} \\quad \\mathbf{0101} $$Step 2: Convert each group to hexadecimal $$ 1101_2 = D_{16}, \\quad 0101_2 = 5_{16} $$Final result: $$ 11010101_2 = D5_{16} $$The step-by-step above becomes even more direct when seen all at once. In the figure, the two 4-bit blocks appear side by side, each with the sum of the weights that produces its hexadecimal digit, and the two digits join into the final result, in the same order the blocks were in:\nEach 4-bit block becomes a hexadecimal digit, and the digits are glued together in the same order as the blocks. Example with other bases:\nExample: Convert \\(2120_{3}\\) to base \\(9\\) Since \\(9 = 3^{2}\\), each digit in base \\(9\\) corresponds to a block of 2 digits in base \\(3\\).\nStep 1: Group the digits in twos, from right to left: $$ 2120_{3} \\quad \\Rightarrow \\quad \\mathbf{21} \\quad \\mathbf{20} $$Step 2: Convert each block (in base \\(3\\)) to a digit (in base \\(9\\)): $$ \\begin{align*} 21_{3} \u0026 = 2\\cdot 3^{1} + 1\\cdot 3^{0} = 6 + 1 = 7_{10} = 7_{9} \\\\ 20_{3} \u0026 = 2\\cdot 3^{1} + 0\\cdot 3^{0} = 6 + 0 = 6_{10} = 6_{9} \\end{align*} $$Final result: \\(2120_{3} = 76_{9}\\)\nCase 2: Larger Base to Smaller Base (\\(s \u003e r\\)) # If \\(s = r^k\\), you can expand each digit of the larger base into \\(k\\) digits of the smaller base. In computing, again the most common case is expanding octal/hexadecimal to binary.\nExample: Conversion from Octal (\\(8\\)) to Binary (\\(2\\)) Since \\(8 = 2^3\\), each octal digit can be written as 3 binary digits.\nStep 1: Replace each octal digit with its 3-bit binary form $$ 753_8 $$We expand: $$ 7_8 = 111_2, \\quad 5_8 = 101_2, \\quad 3_8 = 011_2 $$Final result: $$ 753_8 = 111101011_2 $$ Example: Conversion from Hexadecimal (\\(16\\)) to Binary (\\(2\\)) Since \\(16 = 2^4\\), each hexadecimal digit can be written as 4 binary digits.\nStep 1: Replace each hexadecimal digit with its 4-bit binary form $$ 4F2_{16} $$We expand: $$ 4_{16} = 0100_2, \\quad F_{16} = 1111_2, \\quad 2_{16} = 0010_2 $$Final result: $$ 4F2_{16} = 010011110010_2 $$ Summary of the Strategies When the Bases Are Related by a Power # Conversion Simplified Method Binary \\(\\to\\) Octal (\\(2 \\to 8\\)) Group into blocks of 3 bits Binary \\(\\to\\) Hexadecimal (\\(2 \\to 16\\)) Group into blocks of 4 bits Octal \\(\\to\\) Binary (\\(8 \\to 2\\)) Expand each octal digit into 3 bits Hexadecimal \\(\\to\\) Binary (\\(16 \\to 2\\)) Expand each hexadecimal digit into 4 bits Method Selection Flowchart # To make it easier to choose the right method for converting between bases, here is a summary flowchart:\n\u0026#x2197; flowchart TD Start([Convert from base s\nto base r]) --\u003e CheckDest{r = 10?} %% Caso 1: Destino é Decimal CheckDest --\u003e|Yes| Method1[Method 1: Positional Expansion\nMultiply each digit by the\nweight of its position] Method1 --\u003e Result([Result in Decimal]) %% Caso 2: Origem é Decimal CheckDest --\u003e|No| CheckOrig{s = 10?} CheckOrig --\u003e|Yes| Method2[Method 2: Successive Divisions\nDivide N successively by r\nand read the remainders from bottom to top] Method2 --\u003e Result2([Result in Base r]) %% Caso 3: Bases Arbitrárias (Ponte ou Atalho) CheckOrig --\u003e|No| CheckPower{sk = r?\nOr rk = s?\nE.g.: 2, 8, 16} %% Atalho CheckPower --\u003e|Yes| Method3a[Shortcut: Grouping or Expansion\nReplace digits with bits or\ngroup bits into digits] Method3a --\u003e Result2 %% Ponte Decimal CheckPower --\u003e|No| Method3b[Method 3: Decimal Bridge\n1. Convert s to 10 Expansion\n2. Convert 10 to r Division] Method3b --\u003e Result2 %% Estilização para ficar bonito no blog style Start fill:#f9f,stroke:#333,stroke-width:2px,color:black style Result fill:#ccf,stroke:#333,stroke-width:2px,color:black style Result2 fill:#ccf,stroke:#333,stroke-width:2px,color:black style Method1 fill:#e1f7d5,stroke:#333,color:black style Method2 fill:#e1f7d5,stroke:#333,color:black style Method3a fill:#fff4e6,stroke:#f66,stroke-width:2px,stroke-dasharray: 5 5,color:black style Method3b fill:#e1f7d5,stroke:#333,color:black Visual explanation of the colors used in the chart:\nLight green: The \u0026ldquo;standard\u0026rdquo; methods (mathematical algorithms). Orange/Dashed: The shortcut (a special path). Purple: Start and end points. Practical Representation: Number Bases in Python # In mathematics, we use subscripts (like \\(101_2\\) or \\(75_{16}\\)) to indicate the base. However, when programming, we cannot easily type subscripts. That is why programming languages adopt prefixes to tell the interpreter which base a number is written in.\nSince Python is the main language covered on this site, here is how to represent and manipulate these bases natively:\nLiterals (Writing Numbers in Code) # To write an integer directly in a specific base, we use the digit 0 followed by an identifying letter:\nBinary: Prefix 0b (e.g., 0b1010) Octal: Prefix 0o (e.g., 0o52) Hexadecimal: Prefix 0x (e.g., 0x2F) When you type these values, Python interprets them and stores them internally as ordinary integers.\n# Python understands all of these values as the same number (42) n_dec = 42 n_bin = 0b101010 n_oct = 0o52 n_hex = 0x2A print(n_dec == n_bin == n_oct == n_hex) # Output: True print(n_hex) # Output: 42 (The default print always shows decimal) Conversion Functions # If you need to convert a number to a string formatted in another base, or convert a string to an integer, use the built-in functions:\nFunction Description Example Input Example Output bin(x) Converts an integer to a binary string bin(10) '0b1010' oct(x) Converts an integer to an octal string oct(10) '0o12' hex(x) Converts an integer to a hexadecimal string hex(10) '0xa' int(s, base) Converts a string in base (b) to an integer int('101', 2) 5 Useful notes:\nint(s, base) accepts bases from 2 to 36, plus base=0, which detects the base from the prefix (0b, 0o, 0x) and treats the string as a Python literal. With an explicit base, the prefix is accepted as long as it matches the base: int('0b101', 2) returns 5, int('0o52', 8) returns 42, and int('0x1A', 16) returns 26. What fails is a contradiction between prefix and base: int('0b101', 10) raises ValueError. There is a fun trap in this rule: int('0b101', 16) neither raises an error nor equals 5. It returns 45313, because in base 16 0b101 is not a prefix at all, it is a perfectly valid hexadecimal numeral (\\(\\mathtt{0B101}_{16}\\)). With base=0, literals follow Python\u0026rsquo;s rules: int('101', 0) is 101, but int('010', 0) raises ValueError, since leading zeros are not allowed. Practical usage example:\nvalue = 255 # Formatting for display (using modern f-strings) # :b for binary, :o for octal, :x for lowercase hex, :X for uppercase hex # Use # to include the prefix (0b, 0o, 0x) in the output: {value:#b}, {value:#x}, etc. print(f\u0026#34;Decimal: {value}\u0026#34;) # 255 print(f\u0026#34;Binary: {value:b}\u0026#34;) # 11111111 print(f\u0026#34;Octal: {value:o}\u0026#34;) # 377 print(f\u0026#34;Hex: {value:X}\u0026#34;) # FF print(f\u0026#34;Hex w/ prefix: {value:#x}\u0026#34;) # 0xff # Reading user input (which always comes as a string) user_input = \u0026#34;1A\u0026#34; number = int(user_input, 16) print(number) # Prints 26 Algorithm Exercise: Implementing \u0026ldquo;By Hand\u0026rdquo; # Although Python\u0026rsquo;s built-in functions (int(), bin(), etc.) are highly optimized and should be used in production code, implementing these converters from scratch is an excellent programming-logic exercise for understanding what happens \u0026ldquo;under the hood\u0026rdquo;.\nBelow, we present didactic implementations of the Positional Expansion and Successive Divisions methods.\nWarning This code is purely educational. In real projects, always prefer the language\u0026rsquo;s built-in functions, which are written in C and much faster.\n# Mapping for digits greater than 9 (A=10, B=11, etc.) DIGITS = \u0026#34;0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\u0026#34; def any_base_to_decimal(number_str: str, source_base: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Implements the Positional Expansion method. Converts a numeric string in source_base to an integer (decimal). \u0026#34;\u0026#34;\u0026#34; decimal_result = 0 # We reverse the string to start from power 0 (right to left) # E.g.: \u0026#34;101\u0026#34; reversed becomes \u0026#34;101\u0026#34; -\u0026gt; indices 0, 1, 2 correspond to 2^0, 2^1, 2^2 reversed_number = number_str[::-1].upper() for position, digit_char in enumerate(reversed_number): # Validate BEFORE looking up: a character outside the alphabet (e.g., \u0026#39;#\u0026#39;) would # make DIGITS.index() blow up with a message that helps no one if digit_char not in DIGITS: raise ValueError(f\u0026#34;Character {digit_char} is not a valid digit\u0026#34;) # Find the numeric value of the character (e.g., \u0026#39;A\u0026#39; -\u0026gt; 10) digit_value = DIGITS.index(digit_char) if digit_value \u0026gt;= source_base: raise ValueError(f\u0026#34;Digit {digit_char} is invalid for base {source_base}\u0026#34;) # Accumulate: digit * (base ^ position) decimal_result += digit_value * (source_base ** position) return decimal_result def decimal_to_any_base(decimal_number: int, target_base: int) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Implements the Successive Divisions method. Converts an integer (decimal) to a string in target_base. \u0026#34;\u0026#34;\u0026#34; if decimal_number == 0: return \u0026#34;0\u0026#34; remainders = [] dividend = decimal_number while dividend \u0026gt; 0: # Get the remainder of the division remainder = dividend % target_base # Get the quotient for the next iteration dividend = dividend // target_base # Convert the remainder to a character (e.g., 15 -\u0026gt; \u0026#39;F\u0026#39;) and store it remainders.append(DIGITS[remainder]) # The algorithm requires reading the remainders from bottom to top (last to first) return \u0026#34;\u0026#34;.join(remainders[::-1]) # --- Testing the functions --- # Test 1: Binary to Decimal (Expansion) binary = \u0026#34;1101\u0026#34; dec = any_base_to_decimal(binary, 2) print(f\u0026#34;Expansion: {binary}_2 to decimal is {dec}\u0026#34;) # Output: Expansion: 1101_2 to decimal is 13 # Test 2: Decimal to Hexadecimal (Division) decimal = 47 hexa = decimal_to_any_base(decimal, 16) print(f\u0026#34;Division: {decimal}_10 to hex is {hexa}\u0026#34;) # Output: Division: 47_10 to hex is 2F # Test 3: The Bridge (Base 9 to Base 4) # We use decimal as the intermediary base9_value = \u0026#34;87\u0026#34; step1_decimal = any_base_to_decimal(base9_value, 9) step2_base4 = decimal_to_any_base(step1_decimal, 4) print(f\u0026#34;Conversion: {base9_value}_9 -\u0026gt; {step1_decimal}_10 -\u0026gt; {step2_base4}_4\u0026#34;) # Output: Conversion: 87_9 -\u0026gt; 79_10 -\u0026gt; 1033_4 Summary Table # Everything this article has established, on a single page:\nIdea Rule Example from the article Positional notation \\(N_b = \\sum_i d_i \\cdot b^i\\) \\(1101_2 = 8 + 4 + 0 + 1 = 13_{10}\\) Valid digits \\(0 \\leq d_i \u003c b\\) in base 9 the digit 9 does not exist The base itself always written as 10 \\(10_2 = 2\\), \\(10_8 = 8\\), \\(10_{16} = 16\\) Any base \\(\\to\\) decimal positional expansion \\(2F_{16} = 2 \\times 16 + 15 = 47_{10}\\) Decimal \\(\\to\\) any base successive divisions, remainders from bottom to top \\(45_{10} = 101101_2\\) Base \\(s \\to\\) base \\(r\\) bridge via decimal \\(87_9 = 79_{10} = 1033_4\\) Binary \\(\\to\\) base \\(2^k\\) group \\(k\\) bits, from right to left \\(101011_2 = 53_8\\) (\\(k = 3\\)) Base \\(2^k \\to\\) binary expand each digit into \\(k\\) bits \\(4F2_{16} = 010011110010_2\\) (\\(k = 4\\)) Padding fill with zeros on the left only \\(101111_2 \\to 0010 1111\\) Verification convert back and compare with the original \\(13_{10} \\to D_{16} \\to 13_{10}\\) In Python bin, oct, hex, int(s, base) int('2F', 16) returns 47 Next Steps # At this point, you already have two essential tools: (1) positional expansion, which lets you interpret a number in any base, and (2) successive divisions, which let you build the representation in another base. In addition, when the bases are related by a power (such as \\(2 \\leftrightarrow 8\\) and \\(2 \\leftrightarrow 16\\)), you can convert almost \u0026ldquo;in your head\u0026rdquo; using grouping and expansion.\nIf you really want to consolidate this, practice with the golden rule: always validate the result by doing the inverse conversion. Start with a few exercises (for example, 5 conversions in each direction) and try alternating methods: one day, use positional expansion; the next, use successive divisions; then use only grouping/expansion when it fits.\nIn the next articles, the idea is to move through the topics that show up all the time in Computer Science. First, we will see how hardware performs arithmetic operations (addition and subtraction) in any base. Then we will understand how to represent signed integers (including Two\u0026rsquo;s Complement) and how to deal with the limits of memory (Overflow). Finally, we will get into real numbers and floating-point representation. These topics explain, for example, why overflow, underflow, and subtle differences between \u0026ldquo;the number\u0026rdquo; and \u0026ldquo;what fits in the register\u0026rdquo; exist.\nIf you like, leave in the comments which conversions you get wrong most often (decimal↔binary, binary↔hex, etc.) and I will include a short section of \u0026ldquo;common mistakes + quick checks\u0026rdquo; with additional examples.\nSee you next time!\n","date":"2026-03-23","externalUrl":null,"permalink":"/2026/03/integer-bases-conversions/","section":"Blog","summary":"In any base, the value of the base itself is written 10. We start from positional notation to convert integers between decimal, binary, octal, and hexadecimal, using expansion, successive divisions, bit grouping, and Python’s bin, oct, hex, and int functions.","title":"Number Bases and Integer Conversion: The Complete Guide","type":"posts"},{"content":"","date":"2023-04-02","externalUrl":null,"permalink":"/categories/data-science/","section":"Categories","summary":"","title":"Data Science","type":"categories"},{"content":"","date":"2023-04-02","externalUrl":null,"permalink":"/tags/pandas/","section":"tags","summary":"","title":"pandas","type":"tags"},{"content":"Both NumPy and pandas have the concept of an axis, but they are used in slightly different ways.\nIn NumPy, the axis refers to the dimension along which a given operation is performed. In pandas, the axis parameter has a slightly different meaning: it refers to the direction along which an operation should be performed.\nIn this article, we will see examples and consequences of this concept in each library.\nNumPy # As written at the beginning of the article, in NumPy the axis refers to the dimension along which a given operation is performed. For example, when computing the sum of a NumPy array, you can specify the axis along which the sum should be computed. If the array has shape (3, 4), then the axis parameter can be set to 0 or 1. If axis=0, the sum is computed along the first dimension, resulting in a 1D array of length 4. If axis=1, the sum is computed along the second dimension, resulting in a 1D array of length 3.\nWe will see this example with two dimensions, and then expand to more complex examples with more dimensions.\nTwo dimensions # Let us start by importing the library and creating a two-dimensional array, checking the length of its dimensions with the shape attribute. If you have questions about the concepts of dimension and shape in the context of NumPy, see this article.\nimport numpy as np arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) arr array([[ 1, 2, 3, 4], [ 5, 6, 7, 8], [ 9, 10, 11, 12]]) arr.shape (3, 4) As explained earlier, axis defines along which dimension a given operation will be performed. So let us sum the values along the first dimension:\nsum_axis_0 = np.sum(arr, axis=0) sum_axis_0 array([15, 18, 21, 24]) We got 4 values in a one-dimensional array. Each of them is the sum of the values along the dimension of size 3. Since the original array arr is two-dimensional, you can visualize it more easily by imagining that the values are the sums of each column of a two-dimensional matrix. But do not get too attached to this visualization, because it will not be very useful when we increase the number of dimensions.\nNow the sum along the second dimension:\nsum_axis_1 = np.sum(arr, axis=1) sum_axis_1 array([10, 26, 42]) Here, we have a one-dimensional array of 3 values. Each of them is the sum of the values along the dimension of size 4. Following our visual analogy, you can see it as the sum of each row of our two-dimensional matrix.\nIt is important to note that the axis parameter can be used with many other NumPy methods, not only the sum. For example, when computing the mean, the median, or the standard deviation of an array, the axis parameter can be used to specify along which dimension the computation should be done. This is especially useful when working with high-dimensional arrays, where operations can become complex. Hence the axis parameter is one of the main tools data scientists use to manipulate and transform data in NumPy arrays.\nSlice operations remain valid, so we can also perform operations on parts of our array, indicating the desired axis:\n# Summing the first two rows along axis 0 sum_rows = np.sum(arr[:2, :], axis=0) sum_rows array([ 6, 8, 10, 12]) # Summing all rows of the first and third columns along axis 0 sum_cols = np.sum(arr[:, [0, 2]], axis=0) sum_cols array([15, 21]) Three dimensions # For three dimensions, the line of reasoning is the same, but the rows-and-columns analogy we used above may not help much. Here we will use visual aids to truly grasp the idea of the axis of an operation. First, let us create a 3D array with random integers:\n# Setting a fixed seed so that the article can be reproduced exactly rng = np.random.default_rng(seed=42) # Creating a 3D array with random integers from 0 to 9 arr_3d = rng.integers(low=0, high=9, size=(2, 3, 4), endpoint=True) Checking the shape:\narr_3d.shape (2, 3, 4) Let us see how NumPy presents this array:\narr_3d array([[[0, 7, 6, 4], [4, 8, 0, 6], [2, 0, 5, 9]], [[7, 7, 7, 7], [5, 1, 8, 4], [5, 3, 1, 9]]]) Notice that it is displayed as if there were two sets of data. In fact, we can take each set and check its shape:\narr_3d[0] array([[0, 7, 6, 4], [4, 8, 0, 6], [2, 0, 5, 9]]) arr_3d[0].shape (3, 4) arr_3d[1] array([[7, 7, 7, 7], [5, 1, 8, 4], [5, 3, 1, 9]]) arr_3d[1].shape (3, 4) That is, we have two two-dimensional matrices with 3 rows and 4 columns, arranged along a dimension of length 2. Confusing? See the following image:\nWith the help of the image, what happens if we ask for the sum along axis 0? Well, it is as if we were pushing the front matrix against the back matrix. So it should be the item-by-item sum of one matrix with the other, and the resulting shape should be (3, 4), the shape of each of the matrices:\nsum_axis_0 = arr_3d.sum(axis=0) display(sum_axis_0) display(sum_axis_0.shape) array([[ 7, 14, 13, 11], [ 9, 9, 8, 10], [ 7, 3, 6, 18]]) (3, 4) Check the result and see that it is indeed what was described. And here we already have a first tip that is very useful when analyzing results of very high dimensions, which are hard to visualize: the resulting shape is the original one with the dimension on which the operation is applied removed. In our case, the original array has shape (2, 3, 4), and we are performing the sum along the dimension of length 2, so the resulting shape excludes this dimension, ending up as (3, 4). Keep this tip in mind to analyze the following examples.\nIn the specific case of this sum, the result is the same as explicitly adding each two-dimensional matrix present at each position of the first axis (axis=0):\narr_3d[0] + arr_3d[1] array([[ 7, 14, 13, 11], [ 9, 9, 8, 10], [ 7, 3, 6, 18]]) But, of course, it is more efficient and concise to use the axis notation.\nNow let us modify our example, asking for the sum along axis 1:\nsum_axis_1 = arr_3d.sum(axis=1) display(sum_axis_1) display(sum_axis_1.shape) array([[ 6, 15, 11, 19], [17, 11, 16, 20]]) (2, 4) See that the shape is the expected one according to the tip we gave earlier. Since we are looking at the second dimension, of length 3, the resulting shape is given by the lengths of the remaining dimensions, that is, (2, 4).\nThe values are nothing more than the sums along axis 1, which can be understood as the sums of each column of each two-dimensional matrix, according to the figure shown. Since there are two matrices, each with four columns, we get a 2 x 4 two-dimensional matrix as a result.\nNotice how NumPy\u0026rsquo;s syntax makes this operation much easier, and also performs it efficiently in terms of memory and execution speed. If we were to perform this sum manually, with a function of our own, we would need to iterate over each matrix of our three-dimensional array and, in each matrix, iterate over each column, storing the sums of each one. Just as an exercise, we can create such a function and perform the sum along axis 1:\ndef sum_column(matrix): result = [] for column in range(len(matrix[0])): total = 0 for row in matrix: total += row[column] result.append(total) return result sum_axis_1_function = [] for matrix in arr_3d: sum_axis_1_function.append(sum_column(matrix)) np.array(sum_axis_1_function) array([[ 6, 15, 11, 19], [17, 11, 16, 20]]) The result is the same as the one obtained by NumPy, but the code is certainly harder to maintain and inefficient in terms of memory and speed. This is the beauty of NumPy and its vectorized operations. That is, it is optimized to apply algorithms usually designed for element-by-element operations, like our function above, to whole sets of values. The gain in performance and semantics is significant.\nWe still have one more dimension in our array. Following the same logic, we know that the array resulting from the sum along this axis will have shape (2, 3), since we are looking at the dimension of length 4 of our original array. And, by the figure, it can be understood as the sums of each row of each two-dimensional matrix. Since there are two matrices, each with three rows, we get a 2 x 3 two-dimensional matrix as a result. Checking:\nsum_axis_2 = arr_3d.sum(axis=2) display(sum_axis_2) display(sum_axis_2.shape) array([[17, 18, 16], [28, 18, 18]]) (2, 3) As before, for didactic purposes only, we can verify that this is exactly the same result that would be obtained by creating a function to iterate over each row of each matrix, performing the sum:\ndef sum_row(matrix): result = [] for row in matrix: total = sum(row) result.append(total) return result sum_axis_2_function = [] for matrix in arr_3d: sum_axis_2_function.append(sum_row(matrix)) np.array(sum_axis_2_function) array([[17, 18, 16], [28, 18, 18]]) Again, it is much more efficient to use NumPy\u0026rsquo;s vectorized operations. But I leave the functions in this article to make it easier to understand and visualize the results obtained.\nMore than three dimensions # Up to 3 dimensions, figures and diagrams help us get a visual idea. However, the great power of NumPy is that it lets us work on problems that, conceptually, may involve multiple dimensions. If you understood the logic presented so far, the results for higher dimensions will be quite natural, even though our ability to visualize is hampered by our visual limitation to three dimensions.\nLet us generate an array of integers with 4 dimensions:\narr_4d = rng.integers(low=0, high=9, size=(2, 2, 3, 4), endpoint=True) arr_4d array([[[[7, 6, 4, 8], [5, 4, 4, 2], [0, 5, 8, 0]], [[8, 8, 2, 6], [1, 7, 7, 3], [0, 9, 4, 8]]], [[[6, 7, 7, 1], [3, 4, 4, 0], [5, 1, 7, 6]], [[9, 7, 3, 9], [4, 3, 9, 3], [0, 4, 7, 1]]]]) See that we generated an array of shape (2, 2, 3, 4). This means that the first dimension has length 2, and each position of this dimension stores a three-dimensional array of shape (2, 3, 4). Not by chance, in the way NumPy presents the array we see two clear sets of arrays. And, within each of them, two other sets that we can visualize as two-dimensional arrays of shape (3, 4). Everything we described in this paragraph follows the same line of reasoning we adopted in 3 dimensions; we just expanded it by one additional dimension.\nWe can view each position of the first dimension of our four-dimensional array:\narr_4d[0] array([[[7, 6, 4, 8], [5, 4, 4, 2], [0, 5, 8, 0]], [[8, 8, 2, 6], [1, 7, 7, 3], [0, 9, 4, 8]]]) arr_4d[1] array([[[6, 7, 7, 1], [3, 4, 4, 0], [5, 1, 7, 6]], [[9, 7, 3, 9], [4, 3, 9, 3], [0, 4, 7, 1]]]) Now, let us sum along this dimension, along axis=0:\nsum_axis_0_4d = arr_4d.sum(axis=0) display(sum_axis_0_4d) display(sum_axis_0_4d.shape) array([[[13, 13, 11, 9], [ 8, 8, 8, 2], [ 5, 6, 15, 6]], [[17, 15, 5, 15], [ 5, 10, 16, 6], [ 0, 13, 11, 9]]]) (2, 3, 4) The shape is the result we expect following the tip given earlier in the article and the discussion above. And the values are the result of adding, item by item, the 3D arrays present along the axis.\nFor axis=1 we have:\ndisplay(arr_4d.sum(axis=1)) display(arr_4d.sum(axis=1).shape) array([[[15, 14, 6, 14], [ 6, 11, 11, 5], [ 0, 14, 12, 8]], [[15, 14, 10, 10], [ 7, 7, 13, 3], [ 5, 5, 14, 7]]]) (2, 3, 4) Here it is the dimension where we enter the 3D array present at each position of axis 0. Thus, we are adding, item by item, the two-dimensional matrices in each 3D array.\nFor axis=2:\ndisplay(arr_4d.sum(axis=2)) display(arr_4d.sum(axis=2).shape) array([[[12, 15, 16, 10], [ 9, 24, 13, 17]], [[14, 12, 18, 7], [13, 14, 19, 13]]]) (2, 2, 4) Following the logic, it is the sum of each column (there are 4 columns) within each two-dimensional matrix, within each 3D array (there are 2 arrays), within each position of our 4D array (there are 2 positions).\nFinally, for axis=3:\ndisplay(arr_4d.sum(axis=3)) display(arr_4d.sum(axis=3).shape) array([[[25, 15, 13], [24, 18, 21]], [[21, 11, 19], [28, 19, 12]]]) (2, 2, 3) Similarly, it is the sum of each row (there are 3 rows) within each two-dimensional matrix, within each 3D array (there are 2 arrays), within each position of our 4D array (there are 2 positions).\nIf you have difficulty, read it again, work it out on a piece of paper, or draw diagrams. I did this myself to understand it when I first came across these operations. But once you understand it, you will never again have trouble understanding the result of an operation involving axes in NumPy!\npandas # We have already written on this site about how the basic pandas structures, Series and DataFrames, are built on top of NumPy arrays. However, pandas dataframes are, by definition, designed to be similar to tabular, two-dimensional structures, with columns and rows. So we need to understand how the concept of axis should be understood in this library.\nSimple DataFrames # I believe the simplest example is to turn the two-dimensional array that served as our first NumPy example into a pandas dataframe. As a reminder, the array was:\narr array([[ 1, 2, 3, 4], [ 5, 6, 7, 8], [ 9, 10, 11, 12]]) Turned into a dataframe:\nimport pandas as pd arr_to_df = pd.DataFrame(arr) arr_to_df 0 1 2 3 0 1 2 3 4 1 5 6 7 8 2 9 10 11 12 Let us see the result of asking for the sum along axis=0:\narr_to_df.sum(axis=0, numeric_only=True) 0 15 1 18 2 21 3 24 dtype: int64 Well, this is nothing more than the sum of each column. And the values are the same ones we obtained, as an array, when we summed along axis=0 in the original NumPy array.\nSimilarly, for axis=1:\narr_to_df.sum(axis=1, numeric_only=True) 0 10 1 26 2 42 dtype: int64 This is the sum of each row. And the values are the same ones we obtained, as an array, when we summed along axis=1 in the original NumPy array.\nBut if the results are the same, is there any difference?\nIn this specific case, no. But let us consider a more typical dataframe example that will help us notice some particularities of the library and will serve as a basic model for the next discussions.\nAs said at the beginning of this section, pandas was designed to handle tabular data. This means that columns represent variables, each with a unique name and a specific data type, and rows represent the observations or records in the dataset. Consider the following dataframe, with name, age, and salary for three fictitious people:\ndata = { \u0026#34;Name\u0026#34;: [\u0026#34;Alice\u0026#34;, \u0026#34;Bruno\u0026#34;, \u0026#34;Carlos\u0026#34;], \u0026#34;Age\u0026#34;: [25, 30, 35], \u0026#34;Salary\u0026#34;: [50000, 60000, 70000], } df = pd.DataFrame(data) df Name Age Salary 0 Alice 25 50000 1 Bruno 30 60000 2 Carlos 35 70000 Now our data has more meaning. It is clear that, if we want the mean age and salary of the people, we are looking at the columns, which means axis=0 in pandas:\nmean_axis_0 = df.mean(axis=0, numeric_only=True) mean_axis_0 Age 30.0 Salary 60000.0 dtype: float64 Since the library always expects a tabular format, where each column has several records of the same type, the axis=0 parameter is usually the default in pandas methods. So this would be enough:\ndf.mean(numeric_only=True) Age 30.0 Salary 60000.0 dtype: float64 Those more used to the library probably have the habit of applying a filter selecting the desired column(s). Thus, we could compute the mean only for the Age column, for example:\ndf[\u0026#34;Age\u0026#34;].mean() 30.0 This way, it is explicit that the operation is acting on a column. We could also do it as below, for both columns:\ndf[[\u0026#34;Age\u0026#34;, \u0026#34;Salary\u0026#34;]].mean() Age 30.0 Salary 60000.0 dtype: float64 Of course, we could ask for the mean per row with axis=1:\nmean_axis_1 = df.mean(axis=1, numeric_only=True) mean_axis_1 0 25012.5 1 30015.0 2 35017.5 dtype: float64 But what would be the meaning of these values? I cannot see any. Of course, there may be contexts where applying operations to rows makes sense, and we will see some below, but the idea is to show that, since pandas works with the idea of tabular data, operations are primarily designed to be applied to columns. Just like how we deal with spreadsheets in programs such as Excel.\nFrom the above, the reader may have already inferred that axis=2 or higher values make no sense in pandas. After all, if we are always reduced to tabular data, we will have either columns (axis=0) or rows (axis=1). However, this does not mean that we cannot have more dimensions; we just need to understand what they mean in the context of the library.\nDataFrames with MultiIndex # Just as in the previous section we turned the two-dimensional array studied with NumPy into a dataframe, let us now do the same with our three-dimensional array. The question is how to convey the idea of dimensions. For this we can work with dataframes with multiple indexes.\nindex = pd.MultiIndex.from_product( [range(s) for s in arr_3d.shape], names=[\u0026#34;dim1\u0026#34;, \u0026#34;dim2\u0026#34;, \u0026#34;dim3\u0026#34;] ) df_multi = pd.DataFrame( {\u0026#34;data\u0026#34;: arr_3d.flatten()}, index=index, ) df_multi data dim1 dim2 dim3 0 0 0 0 1 7 2 6 3 4 1 0 4 1 8 2 0 3 6 2 0 2 1 0 2 5 3 9 1 0 0 7 1 7 2 7 3 7 1 0 5 1 1 2 8 3 4 2 0 5 1 3 2 1 3 9 Let us understand what we did above. The values of our array were listed in the column data, but were organized hierarchically into levels with the help of the indexes dim1, dim2, dim3. See how the indexes create the idea of levels, groups, categories, or hierarchy within our dataframe. You will often see these words I listed when studying dataframes with multiple indexes. Notice, also, that although we are in a two-dimensional structure, the indexes add semantic dimensions to our data. This will become more evident in the next examples but, first, let us see some consequences.\nAs said, the indexes create an idea of groups in levels. Thus, we can use the groupby method and group by a given level. And, with the data grouped, pass some function such as sum to add up the data of a given level:\ndf_multi.groupby(level=\u0026#34;dim1\u0026#34;).sum() data dim1 0 51 1 64 df_multi.groupby(level=\u0026#34;dim2\u0026#34;).sum() data dim2 0 45 1 36 2 34 And nothing prevents us from passing more than one level:\ndf_multi.groupby(level=[\u0026#34;dim1\u0026#34;, \u0026#34;dim2\u0026#34;]).sum() data dim1 dim2 0 0 17 1 18 2 16 1 0 28 1 18 2 18 Perhaps this example with generic index names dim1, dim2, and dim3 is too abstract to make clear the potential of what we are doing here. So let us add more semantics to the example, keeping our three-dimensional array as the base:\ndates = pd.date_range(\u0026#34;2022-01\u0026#34;, periods=arr_3d.shape[0], freq=\u0026#34;M\u0026#34;, name=\u0026#34;Date\u0026#34;) stores = [\u0026#34;Store1\u0026#34;, \u0026#34;Store2\u0026#34;, \u0026#34;Store3\u0026#34;] products = [\u0026#34;ProductA\u0026#34;, \u0026#34;ProductB\u0026#34;, \u0026#34;ProductC\u0026#34;, \u0026#34;ProductD\u0026#34;] multi_index = pd.MultiIndex.from_product( [dates, stores, products], names=[\u0026#34;Date\u0026#34;, \u0026#34;Store\u0026#34;, \u0026#34;Product\u0026#34;] ) df_sales = pd.DataFrame({\u0026#34;Sales\u0026#34;: arr_3d.flatten()}, index=multi_index) df_sales Sales Date Store Product 2022-01-31 Store1 ProductA 0 ProductB 7 ProductC 6 ProductD 4 Store2 ProductA 4 ProductB 8 ProductC 0 ProductD 6 Store3 ProductA 2 ProductB 0 ProductC 5 ProductD 9 2022-02-28 Store1 ProductA 7 ProductB 7 ProductC 7 ProductD 7 Store2 ProductA 5 ProductB 1 ProductC 8 ProductD 4 Store3 ProductA 5 ProductB 3 ProductC 1 ProductD 9 The values are exactly the same as in our previous dataframe. However, now we see that there are levels for our data: the product they refer to, the stores of these products, the month the values refer to. And the values represent the total number of units sold of each product.\nThe groupings will now have much more meaning for us. Grouping by date and asking for the sum, we get the total number of units sold per month:\ndf_sales.groupby(level=\u0026#34;Date\u0026#34;).sum() Sales Date 2022-01-31 51 2022-02-28 64 We can break it down by month and store:\ndf_sales.groupby(level=[\u0026#34;Date\u0026#34;, \u0026#34;Store\u0026#34;]).sum() Sales Date Store 2022-01-31 Store1 17 Store2 18 Store3 16 2022-02-28 Store1 28 Store2 18 Store3 18 Compare the results with the one obtained in the NumPy section, when the sum was performed along axis=2.\nWe can also break it down by month and product:\ndf_sales.groupby(level=[\u0026#34;Date\u0026#34;, \u0026#34;Product\u0026#34;]).sum() Sales Date Product 2022-01-31 ProductA 6 ProductB 15 ProductC 11 ProductD 19 2022-02-28 ProductA 17 ProductB 11 ProductC 16 ProductD 20 Compare the results with the one obtained in the NumPy section, when the sum was performed along axis=1.\nDid you manage to follow the comparisons? We obtained the same numerical results, but now with an attached meaning, given by the pandas indexes. It is the indexes that add more dimensions to the dataframe. These are dimensions from the semantic point of view: the dataframe remains a tabular structure of rows and columns.\nWe can do the same with the four-dimensional array we created in the NumPy section:\ndates = pd.date_range(\u0026#34;2022-01\u0026#34;, periods=arr_4d.shape[0], freq=\u0026#34;M\u0026#34;, name=\u0026#34;Date\u0026#34;) stores = [\u0026#34;Store1\u0026#34;, \u0026#34;Store2\u0026#34;, \u0026#34;Store3\u0026#34;] products = [\u0026#34;ProductA\u0026#34;, \u0026#34;ProductB\u0026#34;, \u0026#34;ProductC\u0026#34;, \u0026#34;ProductD\u0026#34;] multi_index = pd.MultiIndex.from_product( [dates, stores, products], names=[\u0026#34;Date\u0026#34;, \u0026#34;Store\u0026#34;, \u0026#34;Product\u0026#34;] ) # Create a DataFrame from the 4D NumPy array and the MultiIndex df_sales_unit_price = pd.DataFrame( arr_4d.reshape(-1, 2), index=multi_index, columns=[\u0026#34;UnitsSold\u0026#34;, \u0026#34;UnitPrice\u0026#34;], ) # Print the DataFrame df_sales_unit_price UnitsSold UnitPrice Date Store Product 2022-01-31 Store1 ProductA 7 6 ProductB 4 8 ProductC 5 4 ProductD 4 2 Store2 ProductA 0 5 ProductB 8 0 ProductC 8 8 ProductD 2 6 Store3 ProductA 1 7 ProductB 7 3 ProductC 0 9 ProductD 4 8 2022-02-28 Store1 ProductA 6 7 ProductB 7 1 ProductC 3 4 ProductD 4 0 Store2 ProductA 5 1 ProductB 7 6 ProductC 9 7 ProductD 3 9 Store3 ProductA 4 3 ProductB 9 3 ProductC 0 4 ProductD 7 1 Observe, now, that we have two columns of values. One indicates the number of units sold of each product, in each store, in each month. The other, the price of each unit of product, in each store, in each month. Think of it as a report from a medium or large company, which has stores with the same products, but in different geographic regions. Thus, each region may have different prices for the same product depending on the supply/demand relations arising from the socioeconomic situation of each region.\nIn this context, a multiplication operation on each row makes sense, to obtain the sales revenue, which is the number of units sold multiplied by the price of each unit. The prod method lets us perform this operation easily:\ndf_sales_unit_price.prod(axis=1) Date Store Product 2022-01-31 Store1 ProductA 42 ProductB 32 ProductC 20 ProductD 8 Store2 ProductA 0 ProductB 0 ProductC 64 ProductD 12 Store3 ProductA 7 ProductB 21 ProductC 0 ProductD 32 2022-02-28 Store1 ProductA 42 ProductB 7 ProductC 12 ProductD 0 Store2 ProductA 5 ProductB 42 ProductC 63 ProductD 27 Store3 ProductA 12 ProductB 27 ProductC 0 ProductD 7 dtype: int64 The return is not a dataframe, as we can verify with the type function:\ntype(_) pandas.core.series.Series What we got is a pandas series. We can assign this series to a column of our original dataframe:\ndf_sales_unit_price[\u0026#34;TotalRevenue\u0026#34;] = df_sales_unit_price.prod(axis=1) df_sales_unit_price UnitsSold UnitPrice TotalRevenue Date Store Product 2022-01-31 Store1 ProductA 7 6 42 ProductB 4 8 32 ProductC 5 4 20 ProductD 4 2 8 Store2 ProductA 0 5 0 ProductB 8 0 0 ProductC 8 8 64 ProductD 2 6 12 Store3 ProductA 1 7 7 ProductB 7 3 21 ProductC 0 9 0 ProductD 4 8 32 2022-02-28 Store1 ProductA 6 7 42 ProductB 7 1 7 ProductC 3 4 12 ProductD 4 0 0 Store2 ProductA 5 1 5 ProductB 7 6 42 ProductC 9 7 63 ProductD 3 9 27 Store3 ProductA 4 3 12 ProductB 9 3 27 ProductC 0 4 0 ProductD 7 1 7 Now, we have in a column information obtained by row-by-row operations. Being in a column, it can be used in groupings. For example, total units and revenue per month:\ndf_sales_unit_price.groupby(level=\u0026#34;Date\u0026#34;)[[\u0026#34;UnitsSold\u0026#34;, \u0026#34;TotalRevenue\u0026#34;]].sum() UnitsSold TotalRevenue Date 2022-01-31 50 238 2022-02-28 64 244 Notice that it does not make sense to include the unit price column, hence the column selection. The grouping could also have been done by store, considering the whole time span of our dataset:\ndf_sales_unit_price.groupby(level=\u0026#34;Store\u0026#34;)[[\u0026#34;UnitsSold\u0026#34;, \u0026#34;TotalRevenue\u0026#34;]].sum() UnitsSold TotalRevenue Store Store1 40 163 Store2 42 213 Store3 32 106 Or, by product:\ndf_sales_unit_price.groupby(level=\u0026#34;Product\u0026#34;)[ [\u0026#34;UnitsSold\u0026#34;, \u0026#34;TotalRevenue\u0026#34;] ].sum() UnitsSold TotalRevenue Product ProductA 23 108 ProductB 42 129 ProductC 25 159 ProductD 24 86 Or, still, by store in each month:\ndf_sales_unit_price.groupby(level=[\u0026#34;Date\u0026#34;, \u0026#34;Store\u0026#34;])[ [\u0026#34;UnitsSold\u0026#34;, \u0026#34;TotalRevenue\u0026#34;] ].sum() UnitsSold TotalRevenue Date Store 2022-01-31 Store1 20 102 Store2 18 76 Store3 12 60 2022-02-28 Store1 20 61 Store2 24 137 Store3 20 46 By product in each month:\ndf_sales_unit_price.groupby(level=[\u0026#34;Date\u0026#34;, \u0026#34;Product\u0026#34;])[ [\u0026#34;UnitsSold\u0026#34;, \u0026#34;TotalRevenue\u0026#34;] ].sum() UnitsSold TotalRevenue Date Product 2022-01-31 ProductA 8 49 ProductB 19 53 ProductC 13 84 ProductD 10 52 2022-02-28 ProductA 15 59 ProductB 23 76 ProductC 12 75 ProductD 14 34 The type of grouping will depend on the interest, as will the aggregation operation (sum, mean, etc.). The focus here is to understand when to use axis=0 or axis=1 and to understand that, in pandas, the idea of higher dimensions is given by the indexes.\nConclusion # Understanding the concept of axis is essential for working with data in libraries such as NumPy and pandas. In NumPy, the axis is used to indicate the dimension along which an operation should be applied in an array. In pandas, which is a library geared toward tabular data, the axis is used for operations on columns or rows.\nWe saw that NumPy can handle arrays of different dimensions, from 1D to 4D or more, while pandas is geared toward 2D tabular data. However, pandas lets us create indexes with multiple levels, which makes it possible to organize data in more dimensions without the need for complex data structures.\nWe hope this article has helped clarify the differences between the concepts of axis in NumPy and pandas and how they can be applied when working with data in Python.\nSee you next time!\n","date":"2023-04-02","externalUrl":null,"permalink":"/2023/04/axis-concept-numpy-pandas/","section":"Blog","summary":"Both NumPy and pandas use the concept of an axis, but with subtle differences. In NumPy, it refers to the dimension along which an operation is performed, while in pandas it indicates the direction of an operation. This article presents examples of using axis in each library.","title":"Understanding the Concept of Axis in NumPy and pandas: How to Work with Data in Python","type":"posts"},{"content":"We learn in math that we can only add matrices of the same order. And such a sum results in a matrix of the same order as those that were added. Matrix multiplication also has a requirement: the number of columns of the first matrix must equal the number of rows of the second matrix. But in computational math problems we sometimes have matrices or vectors of different orders that we would like to add or multiply and that do not meet the requirements just mentioned. In this article we will see how the NumPy broadcasting concept is useful in these contexts and how it relates to vector quantization.\nMathematical definitions # Just to make sure everyone is on the same page, let\u0026rsquo;s quickly lay out the mathematical definitions for matrix addition and multiplication, with an example of each operation.\nTo add two matrices, they must have the same number of rows and columns, and the result will have these same numbers of rows and columns. Mathematically, the sum of a matrix A and a matrix B resulting in a matrix C can be described as:\nIn short, each entry of matrix C can be described as:\nAs an example, see the following sum:\nFor multiplication, the number of columns of the first matrix must equal the number of rows of the second matrix. The resulting matrix has the number of rows of the first and the number of columns of the second matrix. In mathematical notation, we have:\nThe matrices A and B. The product C is:\nThe product C = AB. Or, more succinctly, each entry of matrix C can be described as:\nwith i ranging from 1 to m, and j going from 1 to p, considering that A is an m x n matrix and that B is an n x p matrix.\nHere is an example:\nThe mathematical notation and the examples were taken from Wikipedia.\nNow that we have recalled the definitions, let\u0026rsquo;s move on to Python.\nArrays in pure Python? # If you clicked on this article, you are very likely already slightly familiar with NumPy for numerical computing. Perhaps it is even natural to start by importing the library without thinking much at the beginning of any numerical problem. But let\u0026rsquo;s take a step back and think about the reasons that make NumPy the most widely used library for computing with Python.\nAt first, we could consider using lists. After all, we could put our numerical data in them and, for example, ask for a sum as shown below:\na = [1, 2, 3] b = [1, 2, 3] c = a + b c [1, 2, 3, 1, 2, 3] We already see that the result is not what we expect mathematically. What happens is the concatenation of the lists, not the item-by-item sum. Try multiplication and you will see that it is not even defined between lists. Test with tuples too and you will not get the desired result either.\nThere is yet another reason not to use Python\u0026rsquo;s built-in lists. They accept several types as items of the same list. For example:\n[\u0026#34;a\u0026#34;, 1, [True, False], 2.0] ['a', 1, [True, False], 2.0] Imagine doing numerical operations, or even non-numerical ones, with a list like that. It would make no sense. That is why Python treats lists as a mutable store of items that can be of different types. The \u0026ldquo;+\u0026rdquo; symbol in the first code cell of this article, in the context of lists, only tells the Python interpreter that the two lists must be joined into a single one, and the types of their items make no difference. However, it would be interesting to have some object that could restrict the type of item it can contain.\nHence the concept of arrays in programming. The word \u0026ldquo;array\u0026rdquo; is one of several English terms that are often used in texts even when they are not written in English, and in Portuguese-language material about programming it is common to see it left untranslated. It is worth being clear about its meaning, though. A translation sometimes used for array in Portuguese is vetor (vector). In a context of mathematical operations, that is an acceptable translation. But it is important to know that there is a more general one, valid in any context. Notice that I introduced the term when I was showing the problem of lists accepting several types in their sequence of items. So the best translations of array would be either \u0026ldquo;arranjo\u0026rdquo; or \u0026ldquo;sequência\u0026rdquo; (sequence). And, in the context of programming, a sequence of items of the same type. This last part is the most important. For a good discussion about the translation of this and other terms into Portuguese, I recommend this link.\nKnowing that all items of an array are of the same type is what guarantees predictability in operations. We also get better memory management, but we will not go into that last aspect here.\nWell, now that we know what we are talking about, let\u0026rsquo;s see whether there are built-in implementations of arrays in Python.\nIn the standard Python installation, there is the array module from which we can import the array constructor. As already stressed, the items of an array must be of the same type. In this module, the way to indicate the type is through a type code, a letter placed before the items are passed, which indicates the expected type. For example, by passing the letter u we indicate that it is an array of Unicode characters:\nimport array a = array.array(\u0026#34;u\u0026#34;, [\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;]) a array('u', 'abc') Following the logic, the letter i indicates that the array is of integers:\na = array.array(\u0026#34;i\u0026#34;, [1, 10, 20]) a array('i', [1, 10, 20]) All type codes are listed in the documentation.\nIf we try to pass an item that is not of the indicated type, a TypeError is raised:\n# note how mixing the float 1.0 with the integers 10 and 20 results in an error try: a = array.array(\u0026#34;i\u0026#34;, [1.0, 10, 20]) except TypeError as e: print(repr(e)) TypeError('integer argument expected, got float') Now, our goal here is to do mathematical operations. Does the module handle that naturally?\na = array.array(\u0026#34;i\u0026#34;, [1, 2, 3]) b = array.array(\u0026#34;i\u0026#34;, [1, 2, 3]) c = a + b c array('i', [1, 2, 3, 1, 2, 3]) No\u0026hellip; as we see, Python\u0026rsquo;s built-in array object deals with the problem of different types. But it is not suitable for mathematical operations with numerical arrays. Hence the need to use an external library, NumPy being the most widely used.\nArrays with NumPy. Broadcasting. # Well, let\u0026rsquo;s see right away that NumPy correctly performs the mathematical operation we are using as an example:\nfrom numpy import array a = array([1, 2, 3]) b = array([1, 2, 3]) c = a + b c array([2, 4, 6]) The result is what we expect considering the mathematical definition we saw at the beginning of the article. Now, consider the following cell:\na = array((1, 2, 3)) b = 2 c = a + b c array([3, 4, 5]) The result is intuitive, don\u0026rsquo;t you agree? What NumPy does is, virtually, replicate the value 2 into an array of three equal items and perform the sum. Pay attention to the word virtually. It is not that NumPy actually creates copies in memory (that would be inefficient); it is just an analogy to make it easier to understand. See the following figure to better understand what was written:\nThis replication is called broadcasting, literally \u0026ldquo;casting broadly\u0026rdquo;. The term makes sense given that values are \u0026ldquo;spread\u0026rdquo; until the desired dimension is reached. It is worth remembering that the transmission of messages to scattered receivers in telecommunications uses the same term. For example, the famous broadcaster BBC has its acronym originating from British Broadcasting Corporation. There are interesting discussions about the term broadcasting in programming at this StackOverflow link and about the translation of the term on the Portuguese language StackExchange.\nNow let\u0026rsquo;s start to deepen our discussion. We can describe the operation above as the sum of a scalar, the number 2, to a one-dimensional array.\nRecently, in the article about NumPy reshape, we had some discussions about dimensions of NumPy arrays. The ndim attribute shows that our array a has dimension 1:\na.ndim 1 A similar behavior is observed with a two-dimensional array:\nA = array([[1, 2, 3], [1, 2, 3]]) A array([[1, 2, 3], [1, 2, 3]]) A.ndim 2 C = A + b C array([[3, 4, 5], [3, 4, 5]]) See how the (virtual) replication of the scalar to perform the sum was repeated.\nWe can check what happens in the sum of a two-dimensional array and a one-dimensional one:\nA = array([[1, 2, 3], [1, 2, 3]]) b = array([1, 2, 3]) C = A + b C array([[2, 4, 6], [2, 4, 6]]) Again, we notice that the array b had its row \u0026ldquo;replicated\u0026rdquo; downward so as to allow the sum.\nLimitations of broadcasting # There is no doubt that NumPy broadcasting speeds up numerical operations a lot. However, it is not always possible for the library to carry out this procedure. There are rules that define when broadcasting is possible. Given two arrays, NumPy will compare their shapes starting from the rightmost dimension. Dimensions are compatible when their sizes:\nare equal, or one of them has size 1 If these conditions are not met, the exception ValueError: operands could not be broadcast together is raised.\nThe initial arrays do not need to have the same number of dimensions. And the resulting array will have the same number of dimensions as the initial array with more dimensions, where the size of each dimension is the largest size of the corresponding dimension among the initial arrays. Missing dimensions are considered to have size one.\nSurely the previous paragraphs were quite abstract, so let\u0026rsquo;s go to an example. Consider again A, b and C defined as in the last example and their respective dimension, shape and size:\nfrom numpy import array2string A = array([[1, 2, 3], [1, 2, 3]]) b = array([1, 2, 3]) C = A + b def broadcasting_table(names, arrays, alignment=(5, 4, 6, 4)): columns = (\u0026#34;array\u0026#34;, \u0026#34;ndim\u0026#34;, \u0026#34;shape\u0026#34;, \u0026#34;size\u0026#34;) print(\u0026#34; | \u0026#34;.join([f\u0026#34;{col:^{align}}\u0026#34; for col, align in zip(columns, alignment)])) print(\u0026#34; | \u0026#34;.join([f\u0026#34;{\u0026#39;-\u0026#39;*align}\u0026#34; for align in alignment])) for name, arr in zip(names, arrays): elements = (name, arr.ndim, arr.shape, arr.size) print( \u0026#34; | \u0026#34;.join( [f\u0026#34;{str(el):^{align}}\u0026#34; for el, align in zip(elements, alignment)] ) ) print(\u0026#34;C =\u0026#34;, array2string(C, prefix=\u0026#34;C = \u0026#34;)) print() broadcasting_table((\u0026#34;A\u0026#34;, \u0026#34;b\u0026#34;, \u0026#34;C\u0026#34;), (A, b, C)) C = [[2 4 6] [2 4 6]] | array | ndim | shape | size | | ----- | ---- | ------ | ---- | | A | 2 | (2, 3) | 6 | | b | 1 | (3,) | 3 | | C | 2 | (2, 3) | 6 | If you have difficulty understanding what ndim, shape and size are, see this article here on the site.\nLet\u0026rsquo;s understand the rules step by step considering the result above:\nthe initial arrays are A and b the dimensions are 2 and 1, respectively the rightmost dimension of both has size 3, being equal, allowing broadcasting the next dimension, moving to the left, is 2 in A and 1 in b (by absence), allowing broadcasting the resulting array C has dimension 2 (the larger of A and b), with sizes (2, 3), equal to array A (the largest among the initial ones) The example of an array added to a scalar can be better understood based on the rules we have seen:\na = array((1, 2, 3)) b = 2 c = a + b print(\u0026#34;c =\u0026#34;, array2string(c, prefix=\u0026#34;c = \u0026#34;)) print() broadcasting_table((\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;), (a, array((b,)), c)) c = [3 4 5] | array | ndim | shape | size | | ----- | ---- | ----- | ---- | | a | 1 | (3,) | 3 | | b | 1 | (1,) | 1 | | c | 1 | (3,) | 3 | Comparing with the rules:\nthe initial arrays are a and b the dimensions are 1 and 1, respectively the rightmost dimension of a has size 3, and that of b has size 1, allowing broadcasting the resulting array c has dimension 1 (the larger of a and b), with size 3, equal to array a (the largest among the initial ones) Let\u0026rsquo;s consider an example where broadcasting is not possible. Below, we redefine our arrays and try to perform the sum:\nA = array([ [1, 2, 3], [1, 2, 3]]) b = array([1, 2]) try: C = A + b except ValueError as e: print(repr(e)) ValueError('operands could not be broadcast together with shapes (2,3) (2,) ') Note that the sum was not possible, giving an error that cites the shape of the arrays. The sizes of the rightmost dimensions of each array differ from one another and neither has value 1, so the operation is not possible. Compare each attribute of the arrays in the following table:\nbroadcasting_table((\u0026#34;A\u0026#34;, \u0026#34;b\u0026#34;), (A, b)) | array | ndim | shape | size | | ----- | ---- | ------ | ---- | | A | 2 | (2, 3) | 6 | | b | 1 | (2,) | 2 | Application: vector quantization # This seemingly simple concept of broadcasting is at the heart of one of the most important applications in information technology: vector quantization.\nWhat is quantization? # Quantization is the process of restricting a continuous or large set of values (such as real numbers) to a discrete set (such as integers). For example:\nInput Quantization 0 \u0026lt; x ≤ 1 0.5 1 \u0026lt; x ≤ 2 1.5 2 \u0026lt; x ≤ 3 2.5 In the case above, we have infinite sets of real numbers in each interval restricted to a single value, the center of each interval. These single values are called quantization symbols. Considering that we can expand the table above indefinitely following the logic, if we have an input such as 5.8 we expect 5.5 as output.\nThe example above is one of scalar quantization, since the inputs become a single number that represents the sample.\nLet\u0026rsquo;s create a function that performs scalar quantization, with the help of NumPy\u0026rsquo;s digitize method. digitize returns an array of indices that indicate which interval each input value falls into. We can use these indices to obtain the quantization symbols corresponding to the items of the input array:\nfrom numpy import digitize, linspace, printoptions def scalar_quantization(input_array, cut_points, quantization_symbols): \u0026#34;\u0026#34;\u0026#34; Performs scalar quantization \u0026#34;\u0026#34;\u0026#34; # Finding the indices of the intervals corresponding to each input value quantized_indexes = digitize(input_array, cut_points, right=True) - 1 # Using the indices to obtain the corresponding quantization symbols quantized_array = quantization_symbols[quantized_indexes] return quantized_array # ---------- First example: # Defining the cut points of the intervals cut_points_first_example = [0, 1, 2, 3] # Defining the quantization symbols for each interval quantization_symbols_first_example = array([0.5, 1.5, 2.5]) # Generating an array of input values first_example = linspace(0.001, 3, 10) # Printing the results print(\u0026#34;First example\u0026#34;) with printoptions(precision=3, suppress=True): print(\u0026#34;Input values:\\n\u0026#34;, first_example) print( \u0026#34;Output values after quantization:\\n\u0026#34;, scalar_quantization( first_example, cut_points_first_example, quantization_symbols_first_example ), ) First example Input values: [0.001 0.334 0.667 1.001 1.334 1.667 2. 2.334 2.667 3. ] Output values after quantization: [0.5 0.5 0.5 1.5 1.5 1.5 2.5 2.5 2.5 2.5] Now consider the following table:\nInput Quantization -0.5 \u0026lt; x ≤ 0.5 0 0.5 \u0026lt; x ≤ 1.5 1 1.5 \u0026lt; x ≤ 2.5 2 In this last case, quantization still restricts the interval to the central value. And, following the logic, if we have an input such as 5.8 we expect the value 6 as output. We can apply our function to this second example:\n# ---------- Second example: # Defining the cut points of the intervals cut_points_second_example = [-0.5, 0.5, 1.5, 2.5] # Defining the quantization symbols for each interval quantization_symbols_second_example = array([0, 1, 2]) # Generating an array of input values second_example = linspace(-0.499, 2.5, 10) # Printing the results print(\u0026#34;Second example\u0026#34;) with printoptions(precision=3, suppress=True): print(\u0026#34;Input values:\\n\u0026#34;, second_example) print( \u0026#34;Output values after quantization:\\n\u0026#34;, scalar_quantization( second_example, cut_points_second_example, quantization_symbols_second_example ), ) Second example Input values: [-0.499 -0.166 0.167 0.501 0.834 1.167 1.5 1.834 2.167 2.5 ] Output values after quantization: [0 0 0 1 1 1 2 2 2 2] Notice in both examples that we are taking an input of values and simplifying them into an output that has limits, restrictions. We can think of it as if the input data were being compressed into a particular format. That is exactly the main application of the concept: data compression. And we do not need to be restricted to just one dimension, hence the application of the concept of vectors.\nVector quantization # Vector quantization is a data compression technique widely used in image and signal processing. It is based on the idea of grouping data vectors into clusters and then replacing each vector with the centroid of the nearest cluster. This reduces the amount of data that needs to be stored or transmitted, without greatly affecting the quality of the information.\nIn other words, vector quantization is a data reduction technique, where information is mapped to a smaller number of symbols. It is useful in situations where bandwidth or storage space is limited, or when it is desirable to reduce computational complexity in data processing algorithms.\nTo understand the above, let\u0026rsquo;s consider the following example, adapted and extended from the NumPy documentation.\n200 points are known, corresponding to weight and height values for athletes of 4 sports: basketball, football, gymnastics and marathon. A new point is given, for which we want to know the likely sport of the corresponding athlete. Let\u0026rsquo;s start by visualizing the initial situation, the 200 points. For this simulation of points, we will use Scikit-Learn\u0026rsquo;s make_blobs method.\nfrom sklearn.datasets import make_blobs import matplotlib.pyplot as plt from matplotlib.colors import from_levels_and_colors import matplotlib.image as image from PIL import Image from numpy import arange # defining the plot style and the colors plt.style.use(\u0026#34;seaborn-v0_8-muted\u0026#34;) prop_cycle = plt.rcParams[\u0026#34;axes.prop_cycle\u0026#34;] colors = prop_cycle.by_key()[\u0026#34;color\u0026#34;] # creating the clusters for the classes classes = [\u0026#34;Basketball\u0026#34;, \u0026#34;Football\u0026#34;, \u0026#34;Gymnast\u0026#34;, \u0026#34;Marathon\u0026#34;] X, y_true, centers = make_blobs( n_samples=200, centers=((102, 203), (132, 193), (45, 155), (57, 173)), # starting centroids cluster_std=3.5, random_state=0, shuffle=False, return_centers=True, ) # creating the plot fig, ax = plt.subplots() with Image.open(\u0026#34;images/CP_banner_transp.png\u0026#34;) as logo: logo_width, logo_height = logo.size axin = ax.inset_axes([100,140,45,15], transform=ax.transData) axin.imshow(logo) axin.axis(\u0026#39;off\u0026#39;) cmap, norm = from_levels_and_colors(arange(0, y_true.max() + 2), colors[: len(classes)]) scatter = ax.scatter(X[:, 0], X[:, 1], c=y_true, cmap=cmap, norm=norm) ax.legend(handles=scatter.legend_elements()[0], labels=classes) ax.set_xlabel(\u0026#34;Weight / kg\u0026#34;) ax.set_ylabel(\u0026#34;Height / cm\u0026#34;) ax.set_title(\u0026#34;Height vs weight for 4 athlete classes\u0026#34;) plt.show() See in the plot above that we have already highlighted the four classes of athletes. This classification can be done with an algorithm of the K-means type, which tries to separate samples into groups of equal variance by minimizing the sum of squares within a given cluster.\nWith the clusters created, we can summarize them by their centroids. Then, given a new point, which we will call observation (as if it were an experimental point), we can assign it to a given cluster based on the smallest distance between this new point and the center of each centroid. Note that we are describing exactly a vector quantization, with the creation of the clusters, their centroids and the smallest distance to the centroid. Let\u0026rsquo;s start by enhancing our visualization with the centroids, the point we want to classify and the distance vectors:\nfrom numpy import tile, sqrt, sum fig, ax = plt.subplots() # creating the cluster plot, highlighting the center of each with a black circle base_scatter = ax.scatter(X[:, 0], X[:, 1], c=y_true, cmap=cmap, norm=norm) # the centers are the quantization symbols of our example and were stipulated in # the previous cell, manually, as the start of our simulation centers_scatter = ax.scatter( centers[:, 0], centers[:, 1], c=[\u0026#34;black\u0026#34;] * len(centers), s=200, alpha=0.5, label=\u0026#34;Centers\u0026#34;, ) # creating a red square to highlight our athlete to be classified observation = array((111, 188)) observation_scatter = ax.scatter( *observation, c=[\u0026#34;red\u0026#34;], marker=\u0026#34;s\u0026#34;, s=150, label=\u0026#34;Observation\u0026#34;, zorder=2 ) # computing the distance from each cluster center to our athlete origin = tile(observation, (4, 1)) diff = centers - observation dist = sqrt(sum(diff**2, axis=1)) # creating vectors to represent the distances distance_vectors = ax.quiver( origin[:, 0], origin[:, 1], diff[:, 0], diff[:, 1], units=\u0026#34;xy\u0026#34;, angles=\u0026#34;xy\u0026#34;, scale_units=\u0026#34;xy\u0026#34;, scale=1, color=colors, linewidth=0.5, edgecolor=\u0026#34;black\u0026#34;, ) # finishing the plot with the legend, the axis labels and the titles handles = base_scatter.legend_elements()[0] + ax.get_legend_handles_labels()[0] labels = classes + ax.get_legend_handles_labels()[1] ax.legend(handles=handles, labels=labels) ax.set_xlabel(\u0026#34;Weight / kg\u0026#34;) ax.set_ylabel(\u0026#34;Height / cm\u0026#34;) ax.set_title( \u0026#34;Arrows show distance from observation (athlete to be classified)\u0026#34;, fontsize=10 ) fig.suptitle(\u0026#34;Height vs weight for 4 athlete classes\u0026#34;) with Image.open(\u0026#34;images/CP_banner_transp.png\u0026#34;) as logo: logo_width, logo_height = logo.size axin = ax.inset_axes([100,140,45,15], transform=ax.transData) axin.imshow(logo) axin.axis(\u0026#39;off\u0026#39;) plt.show() Since the shortest vector is the one joining our point to the centroid of the basketball players cluster, then this point probably represents a basketball player. A simple idea, right?\nBut where is the broadcasting here?\nIn the code, we created a simulation from the values of the cluster centers, the centroids, whose values are:\ncenters array([[102, 203], [132, 193], [ 45, 155], [ 57, 173]]) Afterwards, we added the point we want to classify:\nobservation array([111, 188]) Note that the two arrays have different dimensions. But the distance computation first goes through the difference between these two arrays. So broadcasting took place in this operation. The value of the difference was stored in the variable diff according to the code shown earlier:\ndiff array([[ -9, 15], [ 21, 5], [-66, -33], [-54, -15]]) Let\u0026rsquo;s understand why the computation is possible based on the broadcasting rules:\nbroadcasting_table( (\u0026#34;centers\u0026#34;, \u0026#34;observation\u0026#34;, \u0026#34;diff\u0026#34;), (centers, observation, diff), (12, 5, 6, 4) ) | array | ndim | shape | size | | ----------- | ---- | ------ | ---- | | centers | 2 | (4, 2) | 8 | | observation | 1 | (2,) | 2 | | diff | 2 | (4, 2) | 8 | the initial arrays are centers and observation the dimensions are 2 and 1, respectively the rightmost dimension of both has size 2, being equal, allowing broadcasting the next dimension, moving to the left, is 2 in centers and 1 in observation (by absence), allowing broadcasting the resulting array diff has dimension 2 (the larger of the initial arrays), with sizes (4, 2), equal to array centers (the largest among the initial ones) Finally, the distance itself from the point to each centroid is given by the square root of the sum of the squared differences, and these distances were assigned to the variable dist in the code:\nfor class_, distance in zip(classes, dist): print(f\u0026#34;Class: {class_:^11}| Distance: {distance:.2f}\u0026#34;) Class: Basketball | Distance: 17.49 Class: Football | Distance: 21.59 Class: Gymnast | Distance: 73.79 Class: Marathon | Distance: 56.04 Again, since the smallest distance is to the centroid of the basketball players class, we classify the athlete as a basketball player.\nWith this example, we have the basic idea of vector quantization and how it relates to NumPy broadcasting when used in this context. Soon I intend to cover image and signal compression in articles here on the site, which underneath apply the same idea as this example.\nConclusion # In this article, we revisited the concept of array and saw how NumPy broadcasting significantly facilitates mathematical operations with arrays. We saw how broadcasting works in a vector quantization example.\nSee you next time!\n","date":"2023-03-02","externalUrl":null,"permalink":"/2023/03/broadcasting-vector-quantization/","section":"Blog","summary":"We learn in math that there are rules for adding or multiplying matrices. But in computational math problems we sometimes have matrices or vectors that we would like to add or multiply and that do not meet the requirements. In this article we will see how the NumPy broadcasting concept is useful in these contexts and how it relates to vector quantization.","title":"NumPy broadcasting and vector quantization","type":"posts"},{"content":" So, What Is NumPy\u0026rsquo;s reshape? # If you are venturing into data science, you have most likely seen a reshape in some code. Maybe with a mysterious value of -1 in one of the positions, as in reshape(-1, 1). But what does that mean? Why do we sometimes need to transform our data to use model methods in Scikit-Learn? That is what we will see in this article.\nTo try to reproduce a real situation, we will take a dataset, store it in a pandas dataframe, try to apply a Scikit-Learn model directly, and then use reshape to solve the errors we run into.\nImports and Example Dataset # To begin, let\u0026rsquo;s import the packages we need for our study and choose a dataset to use as an example:\nfrom sklearn import datasets from sklearn.linear_model import LinearRegression import numpy as np import pandas as pd The dataset chosen for this article is part of the collection of datasets shipped with the Scikit-Learn package: it contains data on patients with diabetes. Let\u0026rsquo;s get the dataset\u0026rsquo;s variables:\ndiabetes_X, diabetes_y = datasets.load_diabetes(return_X_y=True, as_frame=True) Observe, in the following cells, the first 5 entries of each variable, paying attention to how the results are displayed:\ndiabetes_X.head() age sex bmi bp s1 s2 s3 s4 s5 s6 0 0.038076 0.050680 0.061696 0.021872 -0.044223 -0.034821 -0.043401 -0.002592 0.019908 -0.017646 1 -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412 -0.039493 -0.068330 -0.092204 2 0.085299 0.050680 0.044451 -0.005671 -0.045599 -0.034194 -0.032356 -0.002592 0.002864 -0.025930 3 -0.089063 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038 0.034309 0.022692 -0.009362 4 0.005383 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142 -0.002592 -0.031991 -0.046641 diabetes_y.head() 0 151.0 1 75.0 2 141.0 3 206.0 4 135.0 Name: target, dtype: float64 Although the goal of the article is not to do a detailed study of the dataset, let\u0026rsquo;s at least have a description of what each column means. The base features are in the variable diabetes_X and the target values in the variable diabetes_y. Below is a description taken from the Scikit-Learn documentation:\nage - age in years sex - the person\u0026rsquo;s sex bmi - body mass index (BMI) bp - average blood pressure s1 - tc, total serum cholesterol s2 - ldl, low-density lipoproteins s3 - hdl, high-density lipoproteins s4 - tch, total cholesterol / HDL s5 - ltg, log of serum triglycerides level s6 - glu, blood sugar level The target variable is a measure of disease progression one year after baseline. The Difference Between a Series and a DataFrame # reshape is a method of NumPy, which is one of the base libraries of pandas. So let\u0026rsquo;s start by examining the pandas structures to understand how they are built on top of NumPy.\nTo understand what reshape means, we need to understand what the dimension of a variable is and how to get it. And how it relates to the fact that the variable is a pandas dataframe or a pandas series.\nLet\u0026rsquo;s start with dimensions, using the ndim attribute:\ndiabetes_X.ndim 2 diabetes_y.ndim 1 The dimension 2 for our data in diabetes_X is easy to understand: it is the rows and columns. The dimension 1 for diabetes_y is also intuitive, given that we saw a sequence of numbers, a single dimension.\nThe shape attribute shows the number of items in each dimension:\ndiabetes_X.shape (442, 10) That is, in diabetes_X we have 442 rows and 10 columns.\ndiabetes_y.shape (442,) In diabetes_y we have 442 values.\nLet\u0026rsquo;s look at the type of each variable:\ntype(diabetes_X), type(diabetes_y) (pandas.core.frame.DataFrame, pandas.core.series.Series) We see that diabetes_X is a pandas DataFrame, while diabetes_y is a Series. It is worth describing these pandas entities a little better.\nAccording to the documentation, a DataFrame is a two-dimensional structure with columns of potentially different types. A Series, in turn, is a one-dimensional array capable of holding any type of data. That is, from the documentation alone, we could already know the dimensions of our variables as long as we knew their types.\nThere is yet another attribute of interest for our variables, size, which returns how many elements there are in our structure:\ndiabetes_X.size 4420 Since there are 442 rows and 10 columns in diabetes_X, there are 442 x 10 = 4,420 elements. You can think of it as the cells in a spreadsheet with that many rows and columns.\ndiabetes_y.size 442 For diabetes_y the result is obvious, since it is the number of values in our array.\nCreating Series and DataFrames from a Column # What happens if we take a column of our dataframe? What will its type be?\nbmi_series = diabetes_X[\u0026#34;bmi\u0026#34;] bmi_series.head() 0 0.061696 1 -0.051474 2 0.044451 3 -0.011595 4 -0.036385 Name: bmi, dtype: float64 type(bmi_series) pandas.core.series.Series That is, a DataFrame can be thought of as a container of Series. In fact, in the documentation link given earlier, this very statement appears at the end of the definition of DataFrame.\nSo, when we request the shape we get only one value:\nbmi_series.shape (442,) Now, pay attention to the next cell. Notice that we put an extra pair of brackets around the column name:\nbmi_df = diabetes_X[[\u0026#34;bmi\u0026#34;]] bmi_df.head() bmi 0 0.061696 1 -0.051474 2 0.044451 3 -0.011595 4 -0.036385 type(bmi_df) pandas.core.frame.DataFrame bmi_df.shape (442, 1) See that, with just this change, we now have a DataFrame! A DataFrame with a single column, it is true. But the object is no longer just an array of values, it is a DataFrame.\nTo understand this a little better, let\u0026rsquo;s look at what is underneath pandas: the NumPy library.\nNumPy Underneath pandas # The basic structure of NumPy is the ndarray, a name that means N-dimensional array. We can take our previous variables, of pandas types, and ask for them to be expressed in their most basic types, from NumPy, with the to_numpy method:\ndiabetes_X_array = diabetes_X.to_numpy() diabetes_X_array array([[ 0.03807591, 0.05068012, 0.06169621, ..., -0.00259226, 0.01990842, -0.01764613], [-0.00188202, -0.04464164, -0.05147406, ..., -0.03949338, -0.06832974, -0.09220405], [ 0.08529891, 0.05068012, 0.04445121, ..., -0.00259226, 0.00286377, -0.02593034], ..., [ 0.04170844, 0.05068012, -0.01590626, ..., -0.01107952, -0.04687948, 0.01549073], [-0.04547248, -0.04464164, 0.03906215, ..., 0.02655962, 0.04452837, -0.02593034], [-0.04547248, -0.04464164, -0.0730303 , ..., -0.03949338, -0.00421986, 0.00306441]]) With printoptions, we can view it more conveniently, making the structure of our new variable clearer:\nwith np.printoptions(precision=5, edgeitems=3): display(diabetes_X_array) array([[ 0.03808, 0.05068, 0.0617 , ..., -0.00259, 0.01991, -0.01765], [-0.00188, -0.04464, -0.05147, ..., -0.03949, -0.06833, -0.0922 ], [ 0.0853 , 0.05068, 0.04445, ..., -0.00259, 0.00286, -0.02593], ..., [ 0.04171, 0.05068, -0.01591, ..., -0.01108, -0.04688, 0.01549], [-0.04547, -0.04464, 0.03906, ..., 0.02656, 0.04453, -0.02593], [-0.04547, -0.04464, -0.07303, ..., -0.03949, -0.00422, 0.00306]]) The ndim and shape attributes seen earlier are, in fact, inherited from NumPy:\ntype(diabetes_X_array) numpy.ndarray diabetes_X_array.ndim 2 diabetes_X_array.shape (442, 10) Note, therefore, that the dimension and the shape are derived from the more basic NumPy objects.\nWe can check the same with our target variable:\ndiabetes_y_array = diabetes_y.to_numpy() with np.printoptions(precision=5, edgeitems=3, threshold=10): display(diabetes_y_array) array([151., 75., 141., ..., 132., 220., 57.]) type(diabetes_y_array) numpy.ndarray diabetes_y_array.ndim 1 diabetes_y_array.shape (442,) So we already know that pandas is an extension of NumPy. And reshape? Where does it come into this story?\nScikit-Learn\u0026rsquo;s Dimension Requirements # Many Scikit-Learn methods require us to pass our base variables as a two-dimensional array. Let\u0026rsquo;s see this in our example.\nLet\u0026rsquo;s recall the Series we created from one of the columns of our DataFrame:\nbmi_series = diabetes_X[\u0026#34;bmi\u0026#34;] bmi_series.head() 0 0.061696 1 -0.051474 2 0.044451 3 -0.011595 4 -0.036385 Name: bmi, dtype: float64 type(bmi_series) pandas.core.series.Series bmi_series.shape (442,) And also our target variable:\ndiabetes_y.head() 0 151.0 1 75.0 2 141.0 3 206.0 4 135.0 Name: target, dtype: float64 diabetes_y.ndim 1 diabetes_y.shape (442,) Note\nThe focus of the article is not data science itself, but understanding reshape. So the procedures that follow go straight to the point. In a real project, you should split the data into training and test sets, and do a good exploratory analysis before applying models. Here we will skip this part since it is not our focus.\nWe can try to check how linear the behavior of the target variable is with respect to the \u0026ldquo;bmi\u0026rdquo; Series:\nlr = LinearRegression() lr.fit(bmi_series, diabetes_y) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /tmp/ipykernel_471207/2026011743.py in \u0026lt;module\u0026gt; 1 lr = LinearRegression() ----\u0026gt; 2 lr.fit(bmi_series, diabetes_y) ValueError: Expected 2D array, got 1D array instead: Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample. See that an error occurs, stating Expected 2D array, got 1D array instead, which is already a hint that a Series is not expected, since it is one-dimensional.\nThe last line of the message is:\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample. Let\u0026rsquo;s try to follow the instruction:\nbmi_series.reshape(-1, 1) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /tmp/ipykernel_471207/2203274931.py in \u0026lt;module\u0026gt; ----\u0026gt; 1 bmi_series.reshape(-1, 1) /opt/anaconda/lib/python3.9/site-packages/pandas/core/generic.py in __getattr__(self, name) 5573 ): 5574 return self[name] -\u0026gt; 5575 return object.__getattribute__(self, name) 5576 5577 def __setattr__(self, name: str, value) -\u0026gt; None: AttributeError: 'Series' object has no attribute 'reshape' Notice that the Series type does not have the reshape method. But, as we saw, pandas types are built on top of NumPy types. And reshape is a method of NumPy arrays. So we can do:\nbmi_series_numpy_reshape = bmi_series.to_numpy().reshape(-1, 1) with np.printoptions(precision=5, edgeitems=3, threshold=10): display(bmi_series_numpy_reshape) array([[ 0.0617 ], [-0.05147], [ 0.04445], ..., [-0.01591], [ 0.03906], [-0.07303]]) bmi_series_numpy_reshape.ndim 2 lr.fit(bmi_series_numpy_reshape, diabetes_y) LinearRegression() lr.coef_, lr.intercept_, lr.score(bmi_series_numpy_reshape, diabetes_y) (array([949.43526038]), 152.1334841628967, 0.3439237602253803) There, we managed to run our regression. But we still do not know exactly what reshape does.\nBesides, it would not be very practical, nor Pythonic, to keep resorting to the basic types underneath pandas. There is a better way to approach the question.\nThe Recommended Way # As we saw, using two pairs of brackets, we directly get a DataFrame, which is already a two-dimensional type:\nbmi_df = diabetes_X[[\u0026#34;bmi\u0026#34;]] type(bmi_df) pandas.core.frame.DataFrame bmi_df.shape (442, 1) bmi_df.ndim 2 Now we can pass it directly to the fit method:\nlr.fit(bmi_df, diabetes_y) LinearRegression() lr.coef_, lr.intercept_, lr.score(bmi_df, diabetes_y) (array([949.43526038]), 152.1334841628967, 0.3439237602253803) We get the same result as before, but with much less work.\nAnd, attention, without even needing reshape! Yes, a lot of code uses that method unnecessarily. Great! But what exactly is reshape, and what do the values passed to the method mean?\nUnderstanding the Meaning of (-1, 1) in reshape # Let\u0026rsquo;s create a one-dimensional array and check its shape attribute:\narray_1D = np.array((1, 2, 3)) array_1D array([1, 2, 3]) array_1D.shape (3,) We can change the shape of this array explicitly with reshape, passing the number of rows and columns:\narray_2D = array_1D.reshape(3, 1) array_2D array([[1], [2], [3]]) array_2D.shape (3, 1) array_2D.ndim 2 Now, the same can be done by passing one of the values as -1. That value simply means that NumPy will fill it in based on the total number of elements in the array and the sizes of the other dimensions. That is, if we already say that one of the dimensions has size 1 and there are 3 elements, the remaining dimension can only have size 3:\narray_1D.reshape(-1, 1) array([[1], [2], [3]]) Likewise, if one dimension has size 3, the other can only have size 1:\narray_1D.reshape(3, -1) array([[1], [2], [3]]) Thus, the great advantage of using one of the dimensions as -1 is that the code becomes dynamic. You do not need to know the total number of elements in the array; just say the size of one of the dimensions and reshape does the rest of the work. This is very convenient for the Scikit-Learn example, since we only want to add a second dimension, regardless of the size of our dataset.\nreshape will raise an error if the dimensions passed do not make sense:\narray_1D.reshape(2, 1) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /tmp/ipykernel_471207/2649632558.py in \u0026lt;module\u0026gt; ----\u0026gt; 1 array_1D.reshape(2, 1) ValueError: cannot reshape array of size 3 into shape (2,1) Let\u0026rsquo;s take one more example, creating an array with 2 rows and 4 columns:\narray = np.array([[1, 2, 3, 4], [5, 6, 7, 8]]) array array([[1, 2, 3, 4], [5, 6, 7, 8]]) array.shape (2, 4) With what we have seen, it is easy to ask reshape to turn it into an array with two columns:\narray.reshape(-1, 2) array([[1, 2], [3, 4], [5, 6], [7, 8]]) See that reshape correctly interpreted the -1 passed as 4.\nSince our array has 8 elements, we can have a 3-dimensional array with 2 elements in each. Below, the -1 is interpreted as 2:\narray_3D = array.reshape((-1, 2, 2)) array_3D array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]) We need three indices to get a given element. For example, the first element of the array:\narray_3D[0][0][0] 1 Other Interesting NumPy Methods # Finally, let\u0026rsquo;s recall one of the arrays we created earlier:\narray_2D array([[1], [2], [3]]) This array was created by transforming a one-dimensional array into a two-dimensional one, adding a dimension of size 1. We can remove dimensions of size 1 with the squeeze method:\narray_2D.squeeze() array([1, 2, 3]) Recalling our 8-element array:\narray array([[1, 2, 3, 4], [5, 6, 7, 8]]) We can turn it into a one-dimensional array in two ways:\narray_flatten = array.flatten() array_flatten array([1, 2, 3, 4, 5, 6, 7, 8]) array_ravel = array.ravel() array_ravel array([1, 2, 3, 4, 5, 6, 7, 8]) However, these two ways are not equivalent. In short:\nravel flatten returns only a view (reference) of the original array returns a copy of the original array modifying the resulting array changes the original modifying the resulting array does not change the original faster than flatten slower because it allocates memory library-level function method of the ndarray object How can we check whether an array is a copy of another? We can use the base attribute. If the result is None, we have either the original array or a copy; otherwise, the result is the array that served as its base, its reference:\nprint(array.base) print(array_flatten.base) print(array_ravel.base) None None [[1 2 3 4] [5 6 7 8]] As expected, only ravel returned an array, precisely the array that served as its base, showing that the ravel array is just a view of its base array.\nConclusion # So, did you understand this reshape thing? We saw not only what it means, with examples from its home library NumPy, but also showed the relationship with pandas and with Scikit-Learn models. Now there is no way to be left with doubts when you find this method in some code.\nWhat other method would you like to see detailed here on the site?\nSee you next time!\n","date":"2023-02-02","externalUrl":null,"permalink":"/2023/02/what-is-numpy-reshape/","section":"Blog","summary":"If you are venturing into data science, you have most likely seen a reshape in some code. Maybe with a mysterious value of -1 in one of the positions, as in reshape(-1, 1). But what does that mean? Why do we sometimes need to transform our data to use model methods in Scikit-Learn? That is what we will see in this article.","title":"So, What Is NumPy's reshape?","type":"posts"},{"content":"","date":"2022-08-02","externalUrl":null,"permalink":"/tags/historia/","section":"tags","summary":"","title":"história","type":"tags"},{"content":"","date":"2022-08-02","externalUrl":null,"permalink":"/tags/history/","section":"tags","summary":"","title":"history","type":"tags"},{"content":" Monte Carlo methods are computational algorithms that rely on random samples to obtain numerical results. In this article, we will see how to obtain the value of pi with these methods.\nAn interactive version of the code shown on this page, built with Streamlit, can be used online by clicking the following button:\nStreamlit app Monte Carlo Method # Monte Carlo methods are computational algorithms that rely on random samples to obtain numerical results. The principle is to use randomness in the data to generate results for problems that are, a priori, deterministic.\nA Bit of History # The name Monte Carlo comes from the fact that the idea was developed by the mathematician Stanislaw Ulam while thinking about games of chance, such as those in casinos. It is thus a reference to the Monte Carlo Casino, in Monaco. Ulam was looking for more practical methods to calculate the probability of getting good hands in card games. Readers with some knowledge of statistics and probability may be thinking this is an already solved problem, since combinatorics can be used. However, the calculations can be laborious and time-consuming.\nBelow, the casino that gave the method its name:\nMonte Carlo Casino, in Monaco. Source: Wikimedia Commons Ulam developed his ideas in the mid-1940s. At that time, computers were just beginning to be developed. They were slow, gigantic, and hard to access. Now is the moment to give one more piece of information about Ulam: he worked at the Los Alamos National Laboratory, where nuclear weapons were studied in the famous Manhattan Project, which culminated in the development of the nuclear weapons used in World War II. Thus, he could have access to the most powerful computers of the time, such as the ENIAC.\nIn the photo below, the ENIAC:\nENIAC, one of the first electronic computers. Source: Wikimedia Commons Well, then why was he studying card games? Simple: he was ill and played a lot while recovering. However, he soon realized that the ideas he was developing could be applied to nuclear problems, such as neutron diffusion. Interesting, isn\u0026rsquo;t it?\nCharacteristics # The methods vary, but they usually follow these steps:\n1 Define the domain Define a domain of possible inputs. 2 Generate random inputs Generate inputs randomly from a probability distribution over the domain. 3 Compute Perform deterministic computations on the inputs. 4 Aggregate Aggregate the results. They are used in many scientific problems where analytical solutions are complex or impossible. For example, in kinetic models of gases and fluid dynamics. However, there are many applications in broader scenarios, such as stock portfolio analysis in the financial market.\nA characteristic of such methods is the use of a large number of randomly generated, uniformly distributed values. Hence their growing relevance as computational capacity increased over the years, since more powerful computers are better able to generate pseudorandom numbers.\nFrom what has been described so far, we see that this is a study within inferential statistics, since the goal is to make statements about a universe from a representative set of values (a sample). In this case, random samples, which tend to exhibit the same properties as the population they are drawn from.\nLaw of Large Numbers # In the previous section, we mentioned that Monte Carlo methods use a large number of random values. This characteristic makes the so-called law of large numbers relevant in this context. This law is fundamental in probability theory and states that the arithmetic mean of the results of repeating the same experiment many times tends to approach the expected value as more trials are performed. Mathematically,\n$$ \\lim_{n \\to \\infty} \\frac{1}{n} \\sum_{i=1}^{n} X_i = \\mu $$This law will become evident in the examples shown next.\nComputing the Value of Pi # We recently wrote here on the site about the number pi, showing a bit of the history of how different approximations were used over time. In this article, we will explore another way of obtaining the value, with a Monte Carlo method.\nThe Python code used here can be obtained by clicking the following button, which will redirect you to a GitHub repository:\nchicolucio/pi-approximations Pi approximations calculated using infinite series and Monte Carlo methods Python 2 1 The code can be downloaded as a zip file, or the repository can be cloned using Git.\nAn interactive version can be used online by clicking the following button:\nStreamlit app This interactive version lets you change the parameters (which will be detailed below), as shown in the following animation:\nTo understand the calculation steps, recall that the area of a square is its side squared, and the area of a circle is π times the radius squared.\nTo compute the value of pi, we will consider the following steps:\ndraw a square of unit side and inscribe a quadrant (the circular sector corresponding to a quarter of a circle) in this square the radius of the circle will also be one the area of the square is one and the area of the quadrant is π / 4, so the ratio between the areas is π / 4 uniformly distribute a given number of points in this square count the number of points that fall inside the circular sector the ratio between the number of points inside the circular sector and the total number of points is an estimate of the ratio between the two areas, π / 4 thus, to estimate the value of pi, just multiply the ratio by 4 I imagine this is still quite abstract, so let\u0026rsquo;s visualize it. In the following cell, we will import what is needed to demonstrate the method with the Python language. The PiMonteCarlo class was created by me and its code is in the repository linked above.\nimport matplotlib.pyplot as plt import matplotlib.colors as mcolors import matplotlib.cm as cm import matplotlib.ticker as mtick import numpy as np from src.compute_pi import PiMonteCarlo Let\u0026rsquo;s look at the attributes and methods of this class. The role of each one will be seen throughout the article.\n[x for x in dir(PiMonteCarlo) if not x.startswith(\u0026#39;_\u0026#39;)] ['calculate', 'coords', 'count_inside_quadrant', 'error', 'plot', 'points', 'seed'] The class can be initialized with an integer, which represents the number of points to be uniformly distributed in the unit square. The plot method draws the square with the points:\npi = PiMonteCarlo(100) pi.plot(); Notice in the plot above that there are red and blue points. The red ones represent the points that lie within a distance of 1 unit from the origin of the plot. That is, the points that lie inside the quadrant. To make this easier, we can use the boolean parameter arc of the method to draw the boundary of the quadrant:\npi.plot(arc=True); At the top of the plot appear the number of points, the estimated value of π, and the error of the estimate relative to the known value. However, you may have difficulty seeing how the points relate to a quadrant. Let\u0026rsquo;s increase the number of points:\npi.points = 1000 pi.plot(); Now, with 1000 points, it should be easier to see the quadrant, even without the auxiliary line that delimits it. Let\u0026rsquo;s add the line just to make it even easier:\npi.plot(arc=True); Each point can be described by coordinates, which can be viewed with the coords property. In the next cell, we see the coordinates of the first 10 points:\npi.coords[:10] (Coordinate(x=0.581428494323093, y=0.6131917506200556), Coordinate(x=0.5438194696485, y=0.4252612829893858), Coordinate(x=0.004676373123996713, y=0.8022562182039947), Coordinate(x=0.7927237972419394, y=0.31656658515605185), Coordinate(x=0.3573345769772017, y=0.8554767653880233), Coordinate(x=0.9126151772173419, y=0.8232838228460756), Coordinate(x=0.1376483536535691, y=0.0455611181699167), Coordinate(x=0.7713742399091441, y=0.24378362303324952), Coordinate(x=0.6374898695743463, y=0.491154209189236), Coordinate(x=0.18317750261662535, y=0.7442699731680038)) Now, let\u0026rsquo;s understand the value of the estimate shown above the plot. According to the calculation steps described earlier, we need to count how many of the 1000 points lie within the radius of the quadrant. The count_inside_quadrant method does exactly that:\npi.count_inside_quadrant() 770 The next step is to divide this number by the total number of points and multiply by 4:\n(pi.count_inside_quadrant() / len(pi.coords)) * 4 3.08 Thus, we obtained exactly the value shown above the plot. But, with the class code, we do not need to do this step manually. The calculate property serves this purpose:\npi.calculate 3.08 Internally, calculate does exactly the computation shown in the previous cell. Finally, the error method computes the error of the estimate relative to the currently known value of pi:\npi.error() -0.01960555055392467 Note that this is the value shown above the plot; the only difference is that the value in the plot is presented as a percentage, being 100 times the value in the previous cell.\nWe discussed at the beginning of the article that the estimates tend to get better the more points are used. Notice the word tend: it is a tendency and not a certainty, since this is a random process. We will return to this issue later. First, let\u0026rsquo;s increase the number of points to 10,000:\npi.points = 10_000 pi.plot(); Notice that the quadrant is much more evident now. And the error decreased compared to the previous case.\nAnd with 100,000 points:\npi.points = 100_000 pi.plot(); Again, the quadrant is very evident and the error is smaller.\nMoving on to 1 million points:\npi.points = 1_000_000 pi.plot(); Notice that there is now no significant difference compared to the previous case. In the first part of the article, we saw that there is a tendency toward a value as the number of samples increases. That is what we just witnessed by increasing the number of points in each example.\nWe can formalize this a bit better. In the following cells, we generate 500 example cases, with the number of points ranging from 10 to 1 million. At the end, we generate a plot showing how the percentage error varies as more points are used:\nerrors = [] points = [] for i in np.logspace(1, 6, 500): points.append(int(i)) errors.append(PiMonteCarlo(int(i)).error()) fig, ax = plt.subplots(figsize=(12, 6), facecolor=(1, 1, 1)) norm_colors = mcolors.SymLogNorm(linthresh=0.01, linscale=0.75, vmin=np.min(errors), vmax=np.max(errors)) colormap = cm.seismic_r ax.scatter(points, errors, c=errors, cmap=colormap, norm=norm_colors) ax.semilogx() ax.yaxis.set_major_locator(mtick.MultipleLocator(0.05)) ax.yaxis.set_major_formatter(mtick.PercentFormatter(1)) ax.grid(True, linestyle=\u0026#39;dashed\u0026#39;) ax.set_axisbelow(True) ax.set_title(\u0026#39;Change in error as the number of points increases\u0026#39;, size=18) scalarmappable = cm.ScalarMappable(norm=norm_colors, cmap=colormap) scalarmappable.set_array(errors) colorbar = fig.colorbar(scalarmappable, format=mtick.FuncFormatter(lambda x, pos: \u0026#34;{:.2%}\u0026#34;.format(x)), ticks=mtick.FixedLocator([-0.25, -0.1, -0.05, -0.01, -0.005, 0, 0.005, 0.01, 0.05, 0.1, 0.25])) plt.show() Notice that the plot is symmetric, showing that errors can be positive or negative and tend to occur randomly. Note that, even with few points, there can be simulations with small errors (points with a fainter color). However, this becomes more and more common with more points, indicating that the estimated value gets ever closer to the real one. From 10,000 points on, practically all estimates are already below 1 % error.\nFinally, there is still one attribute of the class that we have not discussed, seed. To make its role more evident, let\u0026rsquo;s reset the number of points to a relatively low value, such as 50:\npi.points = 50 pi.plot(); What happens if we set the number of points to 50 again:\npi.points = 50 pi.plot(); Notice that, even though the requested number of points is the same, their placement changed, as did the estimate of pi. This happens because, internally, the class uses a random generator. More specifically, the NumPy random generator. Thus, even if the requested number of points is the same, they will not necessarily have the same coordinates.\nHowever, usually when we talk about random numbers produced by programs we are, in reality, talking about pseudorandom numbers. That is, numbers that look random but are obtained by algorithms. Such algorithms usually start from what we call a seed. This way, if we initialize a pseudorandom number generator with the same seed, the results obtained will always be the same.\nWhen we created our class, we did not provide a seed value. Thus, this value will change every time. However, we can fix this value:\npi.seed = 42 pi.points = 50 pi.plot(); Now, let\u0026rsquo;s request 50 points again, twice in a row:\npi.points = 50 pi.plot(); pi.points = 50 pi.plot(); Observe that the same points were shown (same positions in the plots) and the same estimates. If we change the seed or reset it by passing None, the coordinates will change:\npi.seed = 0 pi.points = 50 pi.plot(); The subject of random (or pseudorandom) numbers is quite extensive and will be covered in another article in the near future. But keep in mind that, in computing, not everything that looks random really is. The links presented, in particular the one to the NumPy documentation, are great sources of information about it.\nReading Recommendations # The number pi has been studied for centuries, since ancient times, as I already covered in another article. For example, Archimedes made estimates using geometry. For more details on Archimedes\u0026rsquo; work, I recommend the book Arquimedes: uma porta para a ciência (in Portuguese) by Jeanne Bendick, from the incredible Coleção Imortais da Ciência (in Portuguese).\nFor mathematics in general, there is O livro da matemática, the Portuguese edition of DK\u0026rsquo;s The Maths Book, from the series As grandes ideias de todos os tempos (Big Ideas Simply Explained).\nThose who want to go deeper into Monte Carlo methods can buy the book Monte Carlo Statistical Methods.\nFinally, a good book for Python beginners is Automatize tarefas maçantes com Python (in Portuguese, the Brazilian edition of Automate the Boring Stuff with Python). A book with practical, easy-to-read examples.\nConclusion # Monte Carlo methods are very important in the scientific context. In this article, we used visual resources to understand the nature of such methods.\nWe already saw in another article that there are very fast and exact methods to obtain the number pi, so the method presented here is not, in practice, used to estimate pi when great accuracy is needed. However, in other areas of science it turns out to be one of the best alternatives available for estimating quantities, especially with the ever more available computational power today. The visualization presented here is didactic and serves as a reference for other approximations in other contexts.\nSee you next time.\n","date":"2022-08-02","externalUrl":null,"permalink":"/2022/08/monte-carlo-methods-python/","section":"Blog","summary":"Monte Carlo methods are computational algorithms that rely on random samples to obtain numerical results. In this article, we will see how to obtain the value of pi with these methods.","title":"Monte Carlo Methods with Python","type":"posts"},{"content":"","date":"2022-08-02","externalUrl":null,"permalink":"/tags/sympy/","section":"tags","summary":"","title":"SymPy","type":"tags"},{"content":"","date":"2022-07-07","externalUrl":null,"permalink":"/categories/ciencia/","section":"Categorias","summary":"","title":"Ciência","type":"categories"},{"content":"","date":"2022-07-07","externalUrl":null,"permalink":"/categories/science/","section":"Categories","summary":"","title":"Science","type":"categories"},{"content":" The number π is a mathematical constant: the ratio of a circle\u0026rsquo;s circumference to its diameter. It is an irrational number, that is, it cannot be expressed exactly as the ratio of two integers.\nThe animation below shows the definition: a circle of diameter 1, unrolled, measures π.\nA circle of diameter 1, unrolled, measures π. Source: Wikimedia Commons Being irrational means having an infinite decimal representation with no repetition. So, for millennia, mathematicians have tried to improve their understanding of the number π and to obtain its value with greater accuracy.\nIn this article, we will go through the history of the number π and learn a bit of Python programming along the way.\nDefinition # In the introduction we gave a verbal definition of π and showed an animation that helps visualize it. Now, let\u0026rsquo;s move on to a mathematical definition. There are several ways to define π mathematically. Here, we will adopt a definition based on calculus.\nThe circumference of a circle is the length of the arc along its perimeter. Considering, for example, the unit circle given in Cartesian coordinates, we can obtain π as the result of an integral:\n$$ \\text{Unit circle: } \\quad x^{2} + y^{2} = 1 \\\\ \\pi = \\int\\limits_{-1}^{1} \\frac{1}{\\sqrt{1- x^{2}}} dx $$Here at Ciência Programada we have a tag just for articles that use SymPy. Let\u0026rsquo;s see how this Python package handles this integral:\n# setup for better outputs in the article, can be ignored from sympy import init_printing init_printing(scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) from sympy import integrate, Symbol, sqrt, Integral x = Symbol(\u0026#39;x\u0026#39;) expr = 1 / sqrt(1 - x**2) Integral(expr, (x, -1, 1)) \\(\\displaystyle \\int\\limits_{-1}^{1} \\frac{1}{\\sqrt{- x^{2} + 1}} dx\\)\nAbove, we have the representation of the integral. To solve it, we can use the doit method of the Integral object or call the integrate function directly:\nIntegral(expr, (x, -1, 1)).doit() \\(\\displaystyle \\pi\\)\nintegrate(expr, (x, -1, 1)) \\(\\displaystyle \\pi\\)\nNote that the result of the integral is indeed π.\nNow, calculus as we know it today only began to be developed in the 17th century. Before that, how was π defined? Which approximations were used?\nApproximations of π # Throughout history, many approximations of the value of π have been used. A list in chronological order of approximations of π can be found at this link.\nAncient history # The simplest approximation of all is to take the value of π as 3. The Babylonian mathematicians adopted this approximation when appropriate (they knew it was an approximation). And let he who has never used this approximation cast the first stone, engineer :-)\nIn ancient history, many approximations were used by Babylonians, Egyptians, Greeks, Indians\u0026hellip; just to list a few: 25/8, 256/81, 339/108. All of these fractions approximate π with errors smaller than 1 %.\nAround 250 BC, the Greek mathematician Archimedes, using the method of exhaustion, showed that the value of π had to be greater than 223/71 and less than 22/7. In this method, polygons are inscribed in and circumscribed about circles, successively increasing the number of sides in order to come ever closer to the shape of the circle.\nThe figure shows Archimedes\u0026rsquo; polygons:\nArchimedes\u0026rsquo; method: inscribed and circumscribed polygons. Source: Wikimedia Commons For more details on Archimedes\u0026rsquo; work, I recommend the book Arquimedes: uma porta para a ciência (in Portuguese) by Jeanne Bendick, from the incredible Coleção Imortais da Ciência (in Portuguese). Click the links for more details.\nAbout a century later, Ptolemy used the value 377/120, the first known approximation with 3 correct decimal places.\nOver the centuries, Archimedes\u0026rsquo; geometric method was improved and, by the 17th century, such improvements had already brought the value of π to dozens of correct decimal places.\nBut what changed in the 17th century? Infinite series.\nThe revolution of infinite series # We already have an article about infinite series on Ciência Programada. Here we will use a bit of what was covered there, so be sure to read it afterwards ;-)\nAn infinite series is the sum of the terms of an infinite sequence. With equations based on infinite series, a great advance was made in obtaining more precise values for π. Many equations were obtained by many mathematicians, both Eastern and Western. Some equations can be seen here. In this article, we will select just two to illustrate the power of infinite series.\nThe chosen series are presented below. They were chosen for this article for their didactic value. They are relatively simple expressions by very relevant mathematicians, Gottfried Leibniz and Leonhard Euler. In the links for each mathematician, I included articles that tell a bit about the history of each series, for those who are interested; they are worth reading.\n$$ \\begin{aligned} \\pi \u0026= 8 \\sum_{k=0}^{\\infty} \\frac{1}{(4k + 1)(4k + 3)} \u0026\u0026 \\text{Leibniz} \\\\ \\pi \u0026= \\sqrt{6 \\sum_{k=1}^{\\infty} \\frac{1}{k^{2}}} \u0026\u0026 \\text{Euler} \\end{aligned} $$At the beginning of the article, I showed how a symbolic mathematics package like SymPy can be used to represent these summations. My goal now is different: to show how these formulas approach the value of π the more terms are used. I will use very basic Python here, with the standard math library and Matplotlib for plots.\nBoth series can easily be written as functions, as the next cell shows. Note that the bodies of both functions are similar. A variable, result, is initialized to zero and incremented in loops that depend on the desired number of terms. The return value is the final value of that variable after the loop and the mathematical operations shown in the series expressions.\nimport math def pi_leibniz(number_of_terms): result = 0 for k in range(number_of_terms): result += 1/((4 * k + 1) * (4 * k + 3)) return 8 * result def pi_euler(number_of_terms): result = 0 for k in range(1, number_of_terms + 1): result += 1/k**2 return math.sqrt(6 * result) With these functions, we can show graphically that the series tend to the value of π as the number of terms used in the summations increases:\nimport matplotlib.pyplot as plt from mpl_toolkits.axes_grid1.inset_locator import zoomed_inset_axes from mpl_toolkits.axes_grid1.inset_locator import mark_inset plt.style.use(\u0026#39;ggplot\u0026#39;) n = range(1, 101) fig, ax = plt.subplots(figsize=(10, 6)) axins = zoomed_inset_axes(ax, 3, loc=\u0026#39;center\u0026#39;) for axis in (ax, axins): axis.hlines(math.pi, xmin=min(n), xmax=max(n), label=\u0026#39;Value of pi\u0026#39;, linestyle=\u0026#39;--\u0026#39;, color=\u0026#39;red\u0026#39;) axis.plot(n, [pi_leibniz(i) for i in n], label=\u0026#39;Leibniz\u0026#39;, color=\u0026#39;green\u0026#39;) axis.plot(n, [pi_euler(i) for i in n], label=\u0026#39;Euler\u0026#39;, color=\u0026#39;blue\u0026#39;) x1, y1 = 80, 3.12 x2, y2 = 100, 3.15 axins.set_xlim(x1, x2) axins.set_ylim(y1, y2) axins.set_facecolor(\u0026#39;whitesmoke\u0026#39;) mark_inset(ax, axins, loc1=2, loc2=4, fc=\u0026#34;none\u0026#34;, ec=\u0026#34;0.6\u0026#34;) ax.legend() ax.set_ylabel(\u0026#39;Approximate value\u0026#39;) ax.set_xlabel(\u0026#39;Number of terms\u0026#39;) ax.set_title(\u0026#39;Approximation of $\\pi$ by series\u0026#39;) plt.show() Look at the zoomed-in region of the plot: the two series get closer and closer to the reference value.\nIt would be interesting to know the percentage error of each approximation. We can create a function to compute the error of a value given a reference value and build a plot of how the error of each series varies:\ndef error(calculated, expected): return (calculated - expected) / expected from matplotlib.ticker import PercentFormatter, MultipleLocator fig, ax = plt.subplots(figsize=(10, 6)) ax.plot(n[9:], [error(pi_leibniz(i), math.pi) for i in n[9:]], label=\u0026#39;Leibniz\u0026#39;, color=\u0026#39;green\u0026#39;) ax.plot(n[9:], [error(pi_euler(i), math.pi) for i in n[9:]], label=\u0026#39;Euler\u0026#39;, color=\u0026#39;blue\u0026#39;) ax.legend() ax.set_ylabel(\u0026#39;Error / %\u0026#39;) ax.set_xlabel(\u0026#39;Number of terms\u0026#39;) ax.yaxis.set_major_formatter(PercentFormatter(1, symbol=None)) ax.xaxis.set_major_locator(MultipleLocator(10)) ax.set_title(\u0026#39;Approximation of $\\pi$ by series - Percentage error\u0026#39;) plt.show() Note how the Leibniz series quickly gives approximations with less than 1 % error. Euler\u0026rsquo;s needs 30 terms to get into that error range.\nIn theory, we could keep adding terms to get an ever smaller error:\nterms = (100, 500, 1000, 5000) print(\u0026#39;Percentage error x terms\u0026#39;) print(\u0026#39;Terms Leibniz Euler\u0026#39;) for number in terms: error_leibniz = error(pi_leibniz(number), math.pi) * 100 error_euler = error(pi_euler(number), math.pi) * 100 print(f\u0026#39;{number:4} {error_leibniz:.1E} {error_euler:.1E}\u0026#39;) Percentage error x terms Terms Leibniz Euler 100 -1.6E-01 -3.0E-01 500 -3.2E-02 -6.1E-02 1000 -1.6E-02 -3.0E-02 5000 -3.2E-03 -6.1E-03 But what is the limit? How far should we approximate? And, by the way, why get more and more decimal places of π at all?\nWhy know more decimal places of π? # In an article about our atmosphere, I wrote about the importance of not rounding the mathematical constants involved in our calculations. Or, at least, of avoiding drastic rounding, using fewer significant figures than those in our experimental data. This is a bad practice that is propagated in science courses, even at universities. There are well-established standards for rounding and for propagation of uncertainties. Use them.\nBut what about a value like π, which is irrational? How far is it worth going in obtaining more and more decimal places?\nIn the following video, mathematician James Grime explains that 39 decimal places of π are enough to calculate the circumference of the observable universe to the precision of a hydrogen atom.\nIn a recent NASA article, one of the engineers at the agency\u0026rsquo;s Jet Propulsion Laboratory explains that 15 decimal places are used at the institution. Click the link for details, but basically the engineer explains that this is enough for the precision needed in interplanetary navigation.\nAs I write this article, the record for the number of decimal places of π is on the order of trillions! And there are motives for the continued search for more digits:\nit is a way to test supercomputers to evaluate mathematical algorithms to test random number algorithms And, perhaps, the main motive behind the search is the inherent human desire to break records :-)\nHow are values with so many decimal places obtained?\nFast-converging series # The great technological advances and progress in mathematical studies now give us faster ways to get values of π with the desired precision. The series presented earlier had the problem of needing many terms to reach acceptable error margins. So, there is a cost in processing and memory.\nIn the early 20th century, the Indian mathematician Srinivasa Ramanujan developed a rapidly converging formula. That formula was a reference for the Chudnovsky brothers, who, a few decades later, developed another series that produces 14 digits of π per term. Here are the formulas:\n$$ \\begin{aligned} \\frac{1}{\\pi} \u0026= \\frac{2 \\sqrt{2}}{9801} \\sum_{k=0}^{\\infty} \\frac{(4k)! (1103 + 26390 k)}{(k!)^{4} 396^{4k}} \u0026\u0026 \\text{Ramanujan} \\\\ \\frac{1}{\\pi} \u0026= \\frac{12}{640320^{3/2}} \\sum_{k=0}^{\\infty} \\frac{(6k)! (13591409 + 545140134 k)}{(3k)! (k!)^{3} (-640320)^{3k}} \u0026\u0026 \\text{Chudnovsky} \\end{aligned} $$The mathematics behind each formula goes a bit beyond the scope of this article, but those interested can click the links above. Ramanujan\u0026rsquo;s formula made it possible to reach records of millions of digits in the 1980s, and the current records of trillions of digits used Chudnovsky\u0026rsquo;s formula.\nWe can create Python functions to reproduce these formulas, as in the next cell. Notice that the functions are built very similarly to those for the Leibniz and Euler series:\ndef pi_ramanujan(number_of_terms): result = 0 for k in range(number_of_terms): result += (math.factorial(4 * k) * (1103 + 26390 * k)) / (math.factorial(k)**4 * 396**(4*k)) return 1 / ((2 * math.sqrt(2) / 9801) * result) def pi_chudnovsky(number_of_terms): result = 0 for k in range(number_of_terms): result += (math.factorial(6 * k) * (13591409 + 545140134 * k)) / (math.factorial(3*k) * math.factorial(k)**3 * (-640320)**(3*k)) return 1 / ((12 / 640320**(3/2)) * result) Now, we can redo the comparison plot between the series, adding the two new ones:\nfig, ax = plt.subplots(figsize=(10, 6)) axins = zoomed_inset_axes(ax, 3, loc=\u0026#39;center\u0026#39;) for axis in (ax, axins): axis.hlines(math.pi, xmin=min(n), xmax=max(n), label=\u0026#39;Value of pi\u0026#39;, linestyle=\u0026#39;--\u0026#39;, color=\u0026#39;red\u0026#39;) axis.plot(n, [pi_leibniz(i) for i in n], label=\u0026#39;Leibniz\u0026#39;, color=\u0026#39;green\u0026#39;) axis.plot(n, [pi_euler(i) for i in n], label=\u0026#39;Euler\u0026#39;, color=\u0026#39;blue\u0026#39;) axis.plot(n, [pi_ramanujan(i) for i in n], label=\u0026#39;Ramanujan\u0026#39;, color=\u0026#39;orange\u0026#39;) axis.plot(n, [pi_chudnovsky(i) for i in n], label=\u0026#39;Chudnovsky\u0026#39;, color=\u0026#39;purple\u0026#39;) x1, y1 = 80, 3.12 x2, y2 = 100, 3.15 axins.set_xlim(x1, x2) axins.set_ylim(y1, y2) axins.set_facecolor(\u0026#39;whitesmoke\u0026#39;) mark_inset(ax, axins, loc1=2, loc2=4, fc=\u0026#34;none\u0026#34;, ec=\u0026#34;0.6\u0026#34;) ax.legend() ax.set_ylabel(\u0026#39;Approximate value\u0026#39;) ax.set_xlabel(\u0026#39;Number of terms\u0026#39;) ax.set_title(\u0026#39;Approximation of $\\pi$ by series\u0026#39;) plt.show() Note that the two new series cannot even be told apart at the scale of the plot, even in the zoomed region. And correct digits are there from the very beginning of both series, overlapping the reference line.\nAnd how do we know the digits are correct? # Digit extraction methods have been developed that can be used to check which digit sits at a given position. I plan to write about these methods in the future, but here is the link for those interested.\nReading recommendations # For more details on Archimedes\u0026rsquo; work, I recommend the book Arquimedes: uma porta para a ciência (in Portuguese) by Jeanne Bendick, from the incredible Coleção Imortais da Ciência (in Portuguese).\nFor mathematics in general, there is O livro da matemática, the Portuguese edition of DK\u0026rsquo;s The Maths Book, from the series As grandes ideias de todos os tempos (Big Ideas Simply Explained).\nFinally, a good book for Python beginners is Automatize tarefas maçantes com Python (in Portuguese), the Portuguese edition of Automate the Boring Stuff with Python. A book with practical, easy-to-read examples.\nConclusion # In this article we saw how to create Python functions to compute summations. More than that, we saw it in the historical context of the study of one of the most famous numbers in mathematics, π. Using Python, we can clearly see how today\u0026rsquo;s algorithms are more efficient than those developed a few centuries ago. And also how this evolution can be better understood graphically.\nSee you next time.\n","date":"2022-07-07","externalUrl":null,"permalink":"/2022/07/history-of-pi-with-python/","section":"Blog","summary":"The number π is a mathematical constant: the ratio of a circle’s circumference to its diameter. In this article we walk through the history of π and learn a bit of Python programming along the way.","title":"The History of the Number Pi with Python","type":"posts"},{"content":" A good part of the study of calculus is devoted to what are called infinite series. But what, exactly, are infinite series? And how are they related to the paradox of Achilles and the tortoise, one of the so-called paradoxes of Zeno of Elea, the Greek philosopher? What is the connection with quantum mechanics and the uncertainty principle? And where does Python come in? That is what we will find out in this article.\nAchilles and the tortoise # Achilles, a Greek hero, and a tortoise run a race. Since Achilles is faster than the tortoise, the tortoise is given a head start, beginning the race some distance ahead of Achilles\u0026rsquo; starting line. Achilles never overtakes the tortoise, because when he reaches the tortoise\u0026rsquo;s initial position A, it is already farther ahead, at another position B. When Achilles reaches B, the tortoise is no longer there, because it has advanced to a new position C, and so on, ad infinitum.\nThe illustration below shows the successive positions:\nIllustration of the paradox of Achilles and the tortoise. Source: Wikimedia Commons This is one of the so-called paradoxes of Zeno of Elea (c. 495 - c. 430 BC), a Greek philosopher. Clearly, we know that Achilles will pass the tortoise, as a simple matter of everyday experience. But can you find the flaws in the text above? And, especially, can you formulate the problem mathematically?\nZeno was an Eleatic. The Eleatic school of philosophy rejected the validity of sensory and material experience, holding that only logic should be the criterion for truth. The Eleatics relied on the concept of universal unity and held that sensory experiences are not consistent with that unity, being inconsistent and, at times, contradictory. Only thought could overcome false sensory appearances and reach knowledge of being, the fundamental truth that \u0026ldquo;all is one\u0026rdquo;. Moreover, there can be no creation, because being cannot come from non-being, since one thing cannot arise from something that is different from it.\nFaced with this, the natural reaction is this:\nMind blown. Uh-huh, quite a hallucinogenic curious way of thinking. I will not go on with this trip into the details of the school, I leave that to the philosophers. However, the last sentence leads to the idea that change is merely an illusion of the senses, and motion is likewise an illusion, a false appearance of mutability, contrary to universal oneness. Hence the origin of the paradox for Zeno. Let us move on to a more mathematical view.\nInfinite series, the concept of limit, and the solution of the paradox # At its core, the paradox hides the idea of infinity, wouldn\u0026rsquo;t you agree? As described earlier in its statement, we could keep expanding to other positions: D, E, F\u0026hellip; indefinitely. The idea of infinity is not at all trivial and we have already covered it briefly in other articles, to see how it was represented in Python packages such as SymPy.\nAnother idea that, for many centuries, was considered nontrivial and paradoxical is that an infinite sum can produce a finite result. This idea became easier to understand in the 19th century with the concept of limit and the work of Augustin-Louis Cauchy. Let us look at one of his studies:\n$$ \\begin{aligned} a + ax + ax^2 + ax^3 + \\cdots = \\sum_{n=0} ax^n \\\\ \\lim_{n \\to \\infty} \\sum_{n=0} ax^n = \\sum_{n=0}^{\\infty} ax^n = \\frac{a}{1-x} \\qquad 0 \u003c x \u003c 1 \\end{aligned} $$In the first line, we express the idea of an infinite sum with the summation symbol. In the second, we take this sum and put a limit with \\(n\\) tending to infinity, also expressing that limit above the summation symbol. And, for the condition \\(0\u0026lt;x\u0026lt;1\\), it is possible to obtain an expression for the limit. In other words, the limit, which is for an infinite sum, converges to a finite value when this condition is met. Hence the study of convergence of series, so common in calculus courses. We will return to this subject later in the article.\nBut what does this have to do with our paradox?\nTo begin with, let us set some boundary conditions for our problem:\nAchilles starts at an initial position \\(x_A\\) The tortoise starts at an initial position \\(x_T\\) Achilles has a constant speed \\(v_A\\) The tortoise has a constant speed \\(v_T\\) Now, the variation of the position, \\(x\\), of both with time, \\(t\\), can be expressed by the equations of uniform motion studied in physics:\n$$ x(t) = x_0 + vt $$where \\(x_0\\) is the initial position of each one and \\(v\\) is the constant speed of each one.\nWhat are we looking for? Well, we know that, for Achilles to overtake the tortoise, at some position the two will be side by side. How long will this take?\nIf we have the position equations of both, we can set them equal and solve for the time variable. We just need to solve a simple equation, which we have already seen in previous articles with SymPy. Let us start by importing what we need, creating our symbols, and finding the equality that represents the meeting position of both:\n# setup for better outputs in the article, can be ignored from sympy import init_printing init_printing(scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) from sympy import S, symbols, Eq, solve xA, xT, vA, vT, t = symbols(\u0026#39;x_A x_T v_A v_T t\u0026#39;) meeting_position = Eq(xA + vA * t, xT + vT * t) meeting_position \\(\\displaystyle t v_{A} + x_{A} = t v_{T} + x_{T}\\)\nWith the solve function, we can obtain the expression for the time at which such a meeting will occur:\nmeeting_time = solve(meeting_position, t)[0] meeting_time \\(\\displaystyle \\frac{- x_{A} + x_{T}}{v_{A} - v_{T}}\\)\nLet us now put in some values to make the subject more tangible. Let us consider that:\nthe tortoise was given a head start of 100 m Achilles\u0026rsquo; speed is 10 m/s the tortoise\u0026rsquo;s speed is 1/10 m/s The values could be different, I chose these only because they are easier to work out in your head. Substituting into the time expression we obtained, we see that, with a minimum of manipulation, we get an expression compatible with the formula obtained for the limit shown earlier:\n$$ t = \\frac{100}{10 - \\frac{1}{10}} = \\frac{10}{1 - \\frac{1}{100}} $$That is, the time can be expressed by the following infinite series:\n$$ t = 10 + 10 \\cdot \\frac{1}{100} + 10 \\cdot \\frac{1}{10000} + 10 \\cdot \\frac{1}{1000000} + \\cdots = \\frac{1000}{99} $$Substituting the values into the SymPy expression, we get the same value:\nmeeting_time.subs({xA: 0, xT: 100, vA: 10, vT: S(\u0026#39;1/10\u0026#39;)}) # seconds \\(\\displaystyle \\frac{1000}{99}\\)\nWith the n method, we can get a numerical approximation of the time needed, in seconds, for our example:\nmeeting_time.subs({xA: 0, xT: 100, vA: 10, vT: S(\u0026#39;1/10\u0026#39;)}).n() # seconds \\(\\displaystyle 10.1010101010101\\)\nIf we have an answer, where is Zeno\u0026rsquo;s error? Well, there are several approaches, but in our calculations what stands out most is that Zeno did not consider time. Since the speeds are different, the time Achilles takes on each stretch is different from the time the tortoise takes, so that eventually he catches up with, and overtakes, the tortoise. Just note in the series that each term is a smaller increment of time. Remember: we are talking about millennia ago, and concepts such as speed were not yet formally defined. The error in Zeno\u0026rsquo;s statement is the word \u0026ldquo;never\u0026rdquo;, which implies time, whereas the entire statement is formulated in terms of distance traveled. There is also the constant change of frame of reference: Achilles -\u0026gt; tortoise -\u0026gt; Achilles\u0026hellip; which gives the impression of a fresh start at each point. When we fix the reference, as when we defined the equations of uniform motion, this impression disappears.\nAnother important thing to notice is that neither the notion of an infinite series nor that of a limit is needed to resolve the paradox. The simple equations of uniform motion are enough. However, formulating the reasoning via time increments leads to important considerations that use concepts that took centuries to be well defined and established.\nLet us explore the idea of infinite series in SymPy further.\nRepresenting infinite series in SymPy # Let us try to rebuild the summation seen in the paradox example. SymPy has the Sum class, which lets us create a summation:\nfrom sympy import Symbol, oo, Sum, summation # whenever possible, state the constraints of your symbols # here, I will consider the constraints of our previous example a = Symbol(\u0026#39;a\u0026#39;, positive=True) n = Symbol(\u0026#39;n\u0026#39;, positive=True) x = Symbol(\u0026#39;x\u0026#39;, positive=True) expr = a * x**n s = Sum(expr, (n, 0, oo)) s \\(\\displaystyle \\sum_{n=0}^{\\infty} a x^{n}\\)\nSee how easily we created our summation! However, more than creating it, it would be interesting to be able to obtain symbolic results for it. For that, we have the doit method (literally, \u0026ldquo;do it!\u0026rdquo;)\ns.doit() $$\\displaystyle a \\left(\\begin{cases} \\frac{1}{- x + 1} \u0026 \\text{for}\\: x \u003c 1 \\\\ \\sum_{n=0}^{\\infty} x^{n} \u0026 \\text{otherwise} \\end{cases}\\right)$$If you want to skip the step of creating the summation and order it done directly, there is the summation function, which does everything in a single step:\ns = summation(expr, (n, 0, oo)) s $$\\displaystyle a \\left(\\begin{cases} \\frac{1}{- x + 1} \u0026 \\text{for}\\: x \u003c 1 \\\\\\sum_{n=0}^{\\infty} x^{n} \u0026 \\text{otherwise} \\end{cases}\\right)$$How interesting. The answer considers two situations, one with \\(x\u0026lt;1\\) and another where there is no such condition. When \\(x\u0026lt;1\\) (and greater than zero, an assumption omitted because it was set when the symbol was created with positive=True), we get exactly the expression expected from what we saw in the previous section. For other values of \\(x\\) there is no convergence, so SymPy simply leaves the expression indicated.\nLet us, then, substitute the example values used in the previous section:\ns.subs({a: 10, x: S(\u0026#39;1/100\u0026#39;)}) \\(\\displaystyle \\frac{1000}{99}\\)\nAs expected, the same value as our previous calculations. We can ask for a numerical approximation:\ns.subs({a: S(\u0026#39;10\u0026#39;), x: S(\u0026#39;1/100\u0026#39;)}).n() \\(\\displaystyle 10.1010101010101\\)\nThis is the power of a symbolic library. See how, by different paths, we got the same result with few commands. Commands that abstract very elaborate concepts, such as that of an equation or of an infinite series.\nUnderstanding the practical importance of infinite series # The story of the paradox served to introduce the idea of infinite series in a friendlier way. But what are their real applications?\nOne of the articles that gave me the most trouble I most enjoyed writing was the one about integrals. In that article, I show that it is important to be aware that much of what calculators, spreadsheets, and the math functions of programming languages present to you are the result of numerical approximations. For example:\nfrom sympy import exp exp(5) \\(\\displaystyle e^{5}\\)\nSee that SymPy left the exponential indicated. We can ask for a numerical value:\nexp(5).n() # equivalent to exp(5).evalf() \\(\\displaystyle 148.413159102577\\)\nHow does SymPy obtain such a value? We used the exponential function as an example, but the same question applies to trigonometric functions or, more generally, to transcendental functions.\nFrom a computational point of view, everything tends to be reduced to simpler operations, such as additions and multiplications, since they are fast and have been optimized by software and hardware for many decades. There are several algorithms to approximate functions, such as CORDIC, algorithms based on Chebyshev polynomials, and ones based on Taylor series, just to name a few examples. In practice, a combination of several algorithms may be used depending on the input of the function and its domain. Also, which implementation is used may change depending on the hardware, as discussed in this StackOverflow link.\nWe see, therefore, that this is a very extensive subject. In this article, we will first talk about the Taylor series. This series is studied in introductory calculus courses because of its simplicity, and several math libraries have implementations of it, as is the case of SymPy.\nTaylor series # Consider a function that can be expressed by the series\n$$ f(x) = c_0 + c_1(x-a) + c_2(x-a)^2 + c_3(x-a)^3 + \\cdots $$where \\(a\\) is a constant, as are the coefficients \\(c_i\\). Assume that the function has continuous derivatives of all orders. Let us evaluate the function and its derivatives at \\(x=a\\):\n$$ \\begin{aligned} f(x) \u0026= c_0 + c_1(x-a) + c_2(x-a)^2 + c_3(x-a)^3 + \\cdots \u0026\u0026 f(a) = c_0 \\\\ f'(x) \u0026= c_1 + 2c_2(x-a) + 3c_3(x-a)^2 + \\cdots \u0026\u0026 f'(a) = c_1 \\\\ f''(x) \u0026= 2c_2 + 3(2)(1)c_3(x-a) + \\cdots \u0026\u0026 f''(a) = 2!c_2 \\\\ f^n(x) \u0026= n!c_n + (n+1)!c_{n+1}(x-a) + \\cdots \u0026\u0026 f^n(a) = n!c_n \\end{aligned} $$Substituting the coefficients into our original function:\n$$ f(x) = f(a) + f'(a)(x-a) + \\frac{f''(a)}{2!}(x-a)^2 + \\cdots + \\frac{f^n(a)}{n!}(x-a)^n $$which is the so-called Taylor series.\nIn the specific case where \\(a = 0\\), we have:\n$$ f(x) = f(0) + f'(0)(x) + \\frac{f''(0)}{2!}x^2 + \\cdots + \\frac{f^n(0)}{n!}x^n $$which, in some books, is called the Maclaurin series.\nNow, let us recreate all of this in SymPy, that is what you have been reading this for. SymPy has a series function that builds a Taylor series of a function around a given value. It takes a mathematical function, the variable, and the value. Let us create a function with SymPy and the necessary symbols:\nfrom sympy import Function, series f = Function(\u0026#39;f\u0026#39;) a, n, x = symbols(\u0026#39;a n x\u0026#39;) series(f(x), x, a) $$\\displaystyle f{\\left(a \\right)} + \\left(- a + x\\right) \\left. \\frac{d}{d \\xi_{1}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }} + \\frac{\\left(- a + x\\right)^{2} \\left. \\frac{d^{2}}{d \\xi_{1}^{2}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{2} + \\frac{\\left(- a + x\\right)^{3} \\left. \\frac{d^{3}}{d \\xi_{1}^{3}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{6} + \\frac{\\left(- a + x\\right)^{4} \\left. \\frac{d^{4}}{d \\xi_{1}^{4}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{24} + \\frac{\\left(- a + x\\right)^{5} \\left. \\frac{d^{5}}{d \\xi_{1}^{5}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{120} + O\\left(\\left(- a + x\\right)^{6}; x\\rightarrow a\\right)$$Exactly what we expected from what we saw throughout the text, just with a different notation. Note that the last term is presented with a Bachmann-Landau notation, indicating that the remaining terms are of order 6 onward. It is possible to remove this notation with the removeO method:\nseries(f(x), x, a).removeO() $$\\displaystyle \\frac{\\left(- a + x\\right)^{5} \\left. \\frac{d^{5}}{d \\xi_{1}^{5}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{120} + \\frac{\\left(- a + x\\right)^{4} \\left. \\frac{d^{4}}{d \\xi_{1}^{4}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{24} + \\frac{\\left(- a + x\\right)^{3} \\left. \\frac{d^{3}}{d \\xi_{1}^{3}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{6} + \\frac{\\left(- a + x\\right)^{2} \\left. \\frac{d^{2}}{d \\xi_{1}^{2}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{2} + \\left(- a + x\\right) \\left. \\frac{d}{d \\xi_{1}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }} + f{\\left(a \\right)}$$By passing a number as the fourth parameter, we can define the maximum order of the series. For example, passing 3 as the parameter:\nseries(f(x), x, a, 3) $$\\displaystyle f{\\left(a \\right)} + \\left(- a + x\\right) \\left. \\frac{d}{d \\xi_{1}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }} + \\frac{\\left(- a + x\\right)^{2} \\left. \\frac{d^{2}}{d \\xi_{1}^{2}} f{\\left(\\xi_{1} \\right)} \\right|_{\\substack{ \\xi_{1}=a }}}{2} + O\\left(\\left(- a + x\\right)^{3}; x\\rightarrow a\\right)$$Since our function has only one variable, \\(x\\), we can omit it. If we omit the value around which the series expansion is made, it is understood to be zero. Thus:\nseries(f(x)) $$\\displaystyle f{\\left(0 \\right)} + x \\left. \\frac{d}{d \\xi} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }} + \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2} + \\frac{x^{3} \\left. \\frac{d^{3}}{d \\xi^{3}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{6} + \\frac{x^{4} \\left. \\frac{d^{4}}{d \\xi^{4}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{24} + \\frac{x^{5} \\left. \\frac{d^{5}}{d \\xi^{5}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{120} + O\\left(x^{6}\\right)$$We have the Maclaurin series for our function.\nWe can obtain each term of the series individually as part of a tuple with the args attribute:\nseries(f(x)).args $$ \\left( x \\left. \\frac{d}{d \\xi} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}, \\ \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2}, \\ \\frac{x^{3} \\left. \\frac{d^{3}}{d \\xi^{3}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{6}, \\ \\frac{x^{4} \\left. \\frac{d^{4}}{d \\xi^{4}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{24}, \\ \\frac{x^{5} \\left. \\frac{d^{5}}{d \\xi^{5}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{120}, \\ O\\left(x^{6}\\right), \\ f{\\left(0 \\right)}\\right)$$SymPy functions have the taylor_term method, to which we pass the order of the desired term and the variable, and we get the expression of that term:\nf(x).taylor_term(2, x) \\(\\displaystyle \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2}\\)\nAnother way to interact with each term of a series is through the lseries method. This method returns a generator, as we see below. If you do not know what a generator is, see this article about generators.\ngen_f_taylor = f(x).lseries() gen_f_taylor \u0026lt;generator object Expr.series.\u0026lt;locals\u0026gt;.\u0026lt;genexpr\u0026gt; at 0x74d96a121120\u0026gt; With Python\u0026rsquo;s next function we can check each term:\nnext(gen_f_taylor) \\(\\displaystyle f{\\left(0 \\right)}\\)\nnext(gen_f_taylor) \\(\\displaystyle x \\left. \\frac{d}{d \\xi} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}\\)\nnext(gen_f_taylor) \\(\\displaystyle \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2}\\)\nWorking with generators has several advantages. We can use the islice function from the itertools library, present in standard Python installations. We have already written about islice. Basically, we have control over the next outputs of the generator, without needing to keep using the next function over and over. Let us import the function and check its return, passing our generator and asking for the next 5 outputs of the generator:\nfrom itertools import islice islice(gen_f_taylor, 5) \u0026lt;itertools.islice at 0x74d96a1c7f10\u0026gt; Note that the object generated by islice is also a generator. We can ask for the generator to be fully consumed by passing it to some data structure, for example, a tuple with the tuple function:\ntuple(islice(gen_f_taylor, 5)) $$\\displaystyle \\left( \\frac{x^{3} \\left. \\frac{d^{3}}{d \\xi^{3}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{6}, \\ \\frac{x^{4} \\left. \\frac{d^{4}}{d \\xi^{4}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{24}, \\ \\frac{x^{5} \\left. \\frac{d^{5}}{d \\xi^{5}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{120}, \\ \\frac{x^{6} \\left. \\frac{d^{6}}{d \\xi^{6}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{720}, \\ \\frac{x^{7} \\left. \\frac{d^{7}}{d \\xi^{7}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{5040}\\right)$$Let us understand the return we got. We had already started consuming the generator gen_f_taylor, having obtained the first 3 terms of the series. When we passed the value 5 as a parameter to islice, we got the next 5 terms. Since it is an infinite series, we can do this procedure indefinitely, obtaining as many terms as we wish.\nTo start from the beginning of the series, we can redefine our variable or pass the function directly with the lseries method:\ntuple(islice(f(x).lseries(), 5)) $$\\displaystyle \\left( f{\\left(0 \\right)}, \\ x \\left. \\frac{d}{d \\xi} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}, \\ \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2}, \\ \\frac{x^{3} \\left. \\frac{d^{3}}{d \\xi^{3}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{6}, \\ \\frac{x^{4} \\left. \\frac{d^{4}}{d \\xi^{4}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{24}\\right)$$Thus, above, we have the first 5 terms of the series.\nIf we pass 2 numbers to islice, we get the terms from the position of the first number up to the position of the second number. For example, from the third to the seventh term:\ntuple(islice(f(x).lseries(), 3, 7)) $$\\displaystyle \\left( \\frac{x^{3} \\left. \\frac{d^{3}}{d \\xi^{3}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{6}, \\ \\frac{x^{4} \\left. \\frac{d^{4}}{d \\xi^{4}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{24}, \\ \\frac{x^{5} \\left. \\frac{d^{5}}{d \\xi^{5}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{120}, \\ \\frac{x^{6} \\left. \\frac{d^{6}}{d \\xi^{6}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{720}\\right)$$Thus, we have total control over how we consume our series. Again, if you have not worked with generators before, see the articles mentioned. I tried to be very didactic in them, starting from very simple examples.\nFinally, we could get the same generator behavior from the series function simply by passing n=None as one of the arguments:\nseries(f(x), n=None) \u0026lt;generator object Expr.series.\u0026lt;locals\u0026gt;.\u0026lt;genexpr\u0026gt; at 0x74d96a32f790\u0026gt; tuple(islice(series(f(x), n=None), 3)) $$\\displaystyle \\left( f{\\left(0 \\right)}, \\ x \\left. \\frac{d}{d \\xi} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}, \\ \\frac{x^{2} \\left. \\frac{d^{2}}{d \\xi^{2}} f{\\left(\\xi \\right)} \\right|_{\\substack{ \\xi=0 }}}{2}\\right)$$OK, lots of letters and numbers, let us look at some more practical examples.\nExponential function # Let us return to the exponential function example. Let us ask for a Taylor series expansion around \\(x = 0\\):\nseries(exp(x)) \\(\\displaystyle 1 + x + \\frac{x^{2}}{2} + \\frac{x^{3}}{6} + \\frac{x^{4}}{24} + \\frac{x^{5}}{120} + O\\left(x^{6}\\right)\\)\nNote that, by default, the expansion goes up to the term of order 5, making it explicit that the remaining terms are of order 6 onward. We can change this highest-order term. In the next cell, we pass the function, the variable, the value of \\(x\\) around which the expansion is made, and the highest-order term:\nseries(exp(x), x, 0, 8) \\(\\displaystyle 1 + x + \\frac{x^{2}}{2} + \\frac{x^{3}}{6} + \\frac{x^{4}}{24} + \\frac{x^{5}}{120} + \\frac{x^{6}}{720} + \\frac{x^{7}}{5040} + O\\left(x^{8}\\right)\\)\nAnd, as already seen, we can remove the order indication of the remaining terms of the series:\nseries(exp(x), x, 0, 8).removeO() \\(\\displaystyle \\frac{x^{7}}{5040} + \\frac{x^{6}}{720} + \\frac{x^{5}}{120} + \\frac{x^{4}}{24} + \\frac{x^{3}}{6} + \\frac{x^{2}}{2} + x + 1\\)\nRemember, a series is nothing more than an infinite summation. Looking at the result above, we can recognize the general expression of the summation and reproduce it with SymPy:\nfrom sympy import factorial n = Symbol(\u0026#39;n\u0026#39;, integer=True) # important to make clear that it must be an integer exp_expr = x**n / factorial(n) # check that this is really the expression sum_exp = Sum(exp_expr, (n, 0, oo)) sum_exp \\(\\displaystyle \\sum_{n=0}^{\\infty} \\frac{x^{n}}{n!}\\)\nDo you agree with the result?\nNow, we can ask SymPy to carry out the summation:\nsum_exp.doit() \\(\\displaystyle e^{x}\\)\nNote that SymPy recognizes that the result of this summation is the exponential function. Remember, all of this could have been done in one line of code with summation:\nsummation(exp_expr, [n, 0, oo]) \\(\\displaystyle e^{x}\\)\nReturning to the exp(5) example used earlier, let us see what we get when we ask for the substitution of \\(x\\) by 5:\nsummation(exp_expr.subs({x: 5}), [n, 0, oo]) \\(\\displaystyle e^{5}\\)\nThus, if we ask for a numerical approximation coming from the summation, it should equal the one obtained directly from the exp expression:\nsummation(exp_expr.subs({x: 5}), [n, 0, oo]).evalf(), exp(5).evalf() \\(\\displaystyle \\left( 148.413159102577, \\ 148.413159102577\\right)\\)\nAnd it really is! Consistent. But what about substitutions from the Taylor series?\nWe know that the approximation will be better the more terms there are in the series. Let us see this, using the previous value as a reference:\nprint(\u0026#39;n Taylor Difference from exp(5)\u0026#39;) print(\u0026#39;-\u0026#39;*45) for i in range(0, 31, 5): approx = series(exp(x), x, 0, i).removeO().subs({x: 5}).n() approx = float(approx) ref = float(exp(5).n()) print(f\u0026#39;{i:2} {approx:20} {approx-ref:15.4E}\u0026#39;) n Taylor Difference from exp(5) --------------------------------------------- 0 0.0 -1.4841E+02 5 65.375 -8.3038E+01 10 143.6894565696649 -4.7237E+00 15 148.37958007973663 -3.3579E-02 20 148.41310786833833 -5.1234E-05 25 148.41315907883663 -2.3740E-08 30 148.41315910257242 -4.1780E-12 See how the difference really does decrease. However, is dozens of terms reasonable for an acceptable difference?\nHere we have a disadvantage of the Taylor series approach. Note that we are working with series built around \\(x=0\\). In the plot below, we see that even lower orders give a good approximation around that value. However, as we move away, as in the case of \\(x=5\\), the approximation only becomes acceptable with more terms. And, in case you have not noticed yet, for the exponential function it makes no sense to expand around \\(x=5\\), because the derivative of an exponential is the exponential itself. Around zero this problem does not exist. That is why Maclaurin series are the ones that appear most in exercises, since they avoid problems of this kind. We will come back to discussions along these lines later.\nfrom sympy.plotting import plot import matplotlib.pyplot as plt from mpl_toolkits.axes_grid1.inset_locator import zoomed_inset_axes from mpl_toolkits.axes_grid1.inset_locator import mark_inset def move_sympy_plot_to_axes(sympy_plot, plt_ax): \u0026#34;\u0026#34;\u0026#34;Moves a SymPy plot to a Matplotlib axes Adapted from: https://stackoverflow.com/a/46813804/8706250 Parameters ---------- sympy_plot : SymPy plot plt_ax : Matplotlib axes \u0026#34;\u0026#34;\u0026#34; backend = sympy_plot.backend(sympy_plot) backend.ax = plt_ax backend._process_series(backend.parent._series, plt_ax, backend.parent) backend.ax.spines[\u0026#39;right\u0026#39;].set_color(\u0026#39;none\u0026#39;) backend.ax.spines[\u0026#39;bottom\u0026#39;].set_position(\u0026#39;zero\u0026#39;) backend.ax.spines[\u0026#39;top\u0026#39;].set_color(\u0026#39;none\u0026#39;) plt.close(backend.fig) fig, ax = plt.subplots(figsize=(10, 6)) axins = zoomed_inset_axes(ax, 128, loc=\u0026#39;upper center\u0026#39;) # zoom = 128 p_exp = plot(exp(x), (x, -1, 6), show=False, label=r\u0026#39;\\\\(f(x) = exp(x)\\\\)\u0026#39;) move_sympy_plot_to_axes(p_exp, ax) data_exp = p_exp[0].get_points() # axins.plot(*data_exp) points = [] points.append(data_exp) for i in range(0, 31, 5): taylor = series(exp(x), x, 0, i).removeO() p_taylor = plot(taylor, (x, -1, 6), show=False, label=f\u0026#34;Taylor - order {i}\u0026#34;) move_sympy_plot_to_axes(p_taylor, ax) data_taylor = p_taylor[0].get_points() points.append(data_taylor) # axins.plot(*data_taylor) for p in points: axins.plot(*p) x1, y1 = 4.99, 148.0 x2, y2 = 5.005, 148.5 axins.set_xlim(x1, x2) axins.set_ylim(y1, y2) mark_inset(ax, axins, loc1=2, loc2=4, fc=\u0026#34;none\u0026#34;, ec=\u0026#34;0.6\u0026#34;) ax.scatter(5, exp(5).n(), color=\u0026#39;red\u0026#39;, label=\u0026#39;Point (5, exp(5))\u0026#39;) axins.scatter(5, exp(5).n(), color=\u0026#39;red\u0026#39;) ax.legend() ax.set_ylim(0, 150) ax.legend(loc=\u0026#39;upper left\u0026#39;, bbox_to_anchor=(1, 1)) plt.show() Convergence of series # We saw earlier that for a long time the idea that infinite sums can yield finite values was considered paradoxical. And we have already given many examples showing that it is possible to obtain values for some infinite series. Mathematically, such series are convergent. Hence the study of convergence of series and the many convergence criteria and tests seen in calculus courses and books. It is not our place here to go through such tests but, is SymPy able to tell us whether a given sum converges?\nLet us create a summation from an expression:\nexpr = (-1)**(x-1) / x convergent01 = Sum(expr, (x, 1, oo)) convergent01 \\(\\displaystyle \\sum_{x=1}^{\\infty} \\frac{\\left(-1\\right)^{x - 1}}{x}\\)\nSymPy has an is_convergent method that returns a boolean:\nconvergent01.is_convergent() \\(\\displaystyle \\text{True}\\)\nThat is, in this example we have a case that converges. But to what value? Simple, just use the doit method to find out:\nconvergent01.doit() \\(\\displaystyle \\log{\\left(2 \\right)}\\)\nLet us look at a case of divergence:\nexpr = 1/(2 * x) divergent01 = Sum(expr, (x, 1, oo)) divergent01 \\(\\displaystyle \\sum_{x=1}^{\\infty} \\frac{1}{2 x}\\)\ndivergent01.is_convergent() \\(\\displaystyle \\text{False}\\)\ndivergent01.doit() \\(\\displaystyle \\infty\\)\nNote that the is_convergent method returned False, so that, when we asked for the sum to be carried out, the return was infinity and not a specific value. Consistent with a divergent series.\nUsually, in the study of convergent series, the so-called geometric series appear, with the following definition:\nfrom sympy import symbols u, a = symbols(\u0026#39;u a\u0026#39;) geom_series_expr = u**n * a geom_series = Sum(geom_series_expr, (n, 0, oo)) geom_series \\(\\displaystyle \\sum_{n=0}^{\\infty} a u^{n}\\)\nLet us see the result of the doit method:\ngeom_series.doit() $$\\displaystyle a \\left(\\begin{cases} \\frac{1}{- u + 1} \u0026 \\text{for}\\: \\left|{u}\\right| \u003c 1 \\\\\\sum_{n=0}^{\\infty} u^{n} \u0026 \\text{otherwise} \\end{cases}\\right)$$Note that there are two possibilities, which depend on the value of u. Let us make substitutions to illustrate the two possibilities.\n# u \u0026lt; 1, should converge: geom_series.subs({a: 1, u: 1/2}).is_convergent() \\(\\displaystyle \\text{True}\\)\ngeom_series.subs({a: 1, u: 1/2}).doit() \\(\\displaystyle 2\\)\nThus, since \\(u \u0026lt; 1\\), the series converges. In this case, to the value 2.\nLooking at a case of divergence:\n# u \u0026gt; 1, should diverge geom_series.subs({a: 1, u: 3/2}).is_convergent() \\(\\displaystyle \\text{False}\\)\ngeom_series.subs({a: 1, u: 3/2}).doit() \\(\\displaystyle \\infty\\)\nThe series diverges, hence the indication of infinity in the result.\nParity of functions. Pendulums # The parity of functions is a concept linked to symmetry. Even or odd functions can greatly simplify some mathematical manipulations, since patterns appear in terms of series. For more details, see this article about parity of functions. Here, we will see how to recognize whether a function is even or odd from its Taylor series. We will take the opportunity to discuss a few more aspects of these series.\nLet us check the Taylor series of the sine function around \\(\\theta=0\\):\nfrom sympy import sin, cos, diff theta = Symbol(\u0026#39;theta\u0026#39;) series(sin(theta)) \\(\\displaystyle \\theta - \\frac{\\theta^{3}}{6} + \\frac{\\theta^{5}}{120} + O\\left(\\theta^{6}\\right)\\)\nThe sine and cosine functions can be related through their derivatives. We have already written about derivatives in SymPy:\ndiff(sin(theta), theta, 1), diff(cos(theta), theta, 1) # first derivative of each function \\(\\displaystyle \\left( \\cos{\\left(\\theta \\right)}, \\ - \\sin{\\left(\\theta \\right)}\\right)\\)\nTherefore, in the Taylor series of both functions there must be an alternation of sine and cosine. Let us check:\nprint(\u0026#39; Sine function\u0026#39;) print(\u0026#39;n d^n(f(x))/dx d^n(f(x))/dx|x=0 Term\u0026#39;) print(\u0026#39;-\u0026#39;*45) for i in range(8): print(f\u0026#39;{i} {diff(sin(theta), theta, i)!s:^15} {diff(sin(theta), theta, i).subs(theta, 0)!s:^15} {sin(theta).taylor_term(i, theta)!s:^15}\u0026#39;) Sine function n d^n(f(x))/dx d^n(f(x))/dx|x=0 Term --------------------------------------------- 0 sin(theta) 0 0 1 cos(theta) 1 theta 2 -sin(theta) 0 0 3 -cos(theta) -1 -theta**3/6 4 sin(theta) 0 0 5 cos(theta) 1 theta**5/120 6 -sin(theta) 0 0 7 -cos(theta) -1 -theta**7/5040 We see that in the series, expanded around zero, only terms with odd exponents appear. This is a characteristic of odd functions. One of the many properties of such functions is that \\(f(-x) = -f(x)\\).\nLet us look at the cosine function:\nseries(cos(theta)) \\(\\displaystyle 1 - \\frac{\\theta^{2}}{2} + \\frac{\\theta^{4}}{24} + O\\left(\\theta^{6}\\right)\\)\nprint(\u0026#39; Cosine function\u0026#39;) print(\u0026#39;n d^n(f(x))/dx d^n(f(x))/dx|x=0 Term\u0026#39;) print(\u0026#39;-\u0026#39;*45) for i in range(8): print(f\u0026#39;{i} {diff(cos(theta), theta, i)!s:^15} {diff(cos(theta), theta, i).subs(theta, 0)!s:^15} {cos(theta).taylor_term(i, theta)!s:^15}\u0026#39;) Cosine function n d^n(f(x))/dx d^n(f(x))/dx|x=0 Term --------------------------------------------- 0 cos(theta) 1 1 1 -sin(theta) 0 0 2 -cos(theta) -1 -theta**2/2 3 sin(theta) 0 0 4 cos(theta) 1 theta**4/24 5 -sin(theta) 0 0 6 -cos(theta) -1 -theta**6/720 7 sin(theta) 0 0 We see that in the series only terms with even exponents appear. This is a characteristic of even functions. One of the many properties of such functions is that \\(f(-x) = f(x)\\).\nLet us also show graphically how more terms in the Taylor series widen the range of the domain in which the approximation is acceptable:\nfrom sympy import pi from matplotlib.ticker import FuncFormatter, MultipleLocator fig, ax = plt.subplots(figsize=(10, 6)) p_sin = plot(sin(theta), (theta, -2*pi, 2*pi), show=False, label=r\u0026#39;\\\\(f(\\theta) = sin(\\theta)\\\\)\u0026#39;) move_sympy_plot_to_axes(p_sin, ax) for i in range(0, 13, 2): taylor = series(sin(theta), theta, 0, i).removeO() p_taylor = plot(taylor, (theta, -2*pi, 2*pi), show=False, label=f\u0026#34;Taylor - order {i}\u0026#34;) move_sympy_plot_to_axes(p_taylor, ax) ax.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;) def format_func(value, tick_number): \u0026#34;\u0026#34;\u0026#34;find number of multiples of pi/2\u0026#34;\u0026#34;\u0026#34; N = int(round(2 * value / pi.n())) if N == 0: return \u0026#34;0\u0026#34; elif N == 1: return r\u0026#34;\\\\(\\dfrac{1}{2}\\pi\\\\)\u0026#34; elif N == 2: return r\u0026#34;\\\\(\\pi\\\\)\u0026#34; elif N % 2 \u0026gt; 0: return r\u0026#34;\\\\(\\dfrac{{{0}}}{{2}}\\pi\\\\)\u0026#34;.format(N) else: return r\u0026#34;\\\\({0}\\pi\\\\)\u0026#34;.format(N // 2) ax.xaxis.set_major_locator(plt.MultipleLocator(float(pi.n()) / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(float(pi.n()) / 4)) ax.xaxis.set_major_formatter(plt.FuncFormatter(format_func)) ax.set_xlabel(r\u0026#39;\\\\(\\theta\\\\)\u0026#39;) ax.set_ylabel(r\u0026#39;\\\\(\\sin(\\theta)\\\\)\u0026#39;) ax.set_ylim(-3, 3) ax.legend() ax.legend(loc=\u0026#39;upper left\u0026#39;, bbox_to_anchor=(1, 1)) plt.show() The series was built around \\(\\theta=0\\). Thus, even with few terms we have an acceptable approximation near this value. However, the farther from this value, the larger the error with few terms. Even with more than 10 terms we still do not have an acceptable approximation at the extremes of the domain shown in the figure. It is very important to be aware of this fact.\nIf you are still having trouble connecting the subject to something more practical, let us look at a common example from physics classes.\nYou probably studied pendulums in your high school physics classes:\nBut, in high school, you probably did not see the full equation that represents the motion of the pendulum over time:\n$$ \\frac{d^2 \\theta}{dt^2} + \\frac{g}{l} \\sin(\\theta) = 0 $$After all, the beauty of differential equations is not shown to our young people :-/ Derivation of the equation at this link about pendulums.\nIn the equation, \\(g\\) is the gravitational acceleration; \\(l\\) is the length of the pendulum\u0026rsquo;s string; and \\(\\theta\\) is the angle of the string with the vertical.\nWhat was probably presented to you is that the variation of the angle and the period are given by:\n$$ \\theta(t) = \\theta_0 \\cos \\left( \\sqrt{\\frac{g}{l}}t \\right) \\\\ T_0 = 2 \\pi \\sqrt{\\frac{l}{g}} $$However, if you later studied differential equations, you will realize that these formulas are not directly obtained from the differential equation. What is often omitted from the student, especially in high school, is that the formulas are only valid for very small angles, usually taken as \\(\\theta \\ll 1\\) radian. Now, such an angle is very close to zero and, therefore, we can take few terms of our Taylor series. In fact, we can take just the first term:\n$$ \\sin(\\theta) \\approx \\theta \\qquad \\theta \\ll 1 $$If this condition holds, we have\n$$ \\frac{d^2 \\theta}{dt^2} + \\frac{g}{l} \\theta = 0 $$A much simpler equation, from which the formulas presented earlier can be obtained. Do it as an exercise for practice :-)\nNotice how the work of solving becomes much simpler with the approximation coming from the Taylor series. Hence the importance of studying it.\nFourier series. Fourier transforms # An article about series could not leave out Fourier series and transforms. Fourier transforms are very common in physics and chemistry, consisting of a mathematical manipulation that reorganizes information. You can imagine, for example, that the human eye and ear perform Fourier transforms by analyzing electromagnetic and sound waves, respectively. The subject is quite broad and will certainly appear more times here on the site. Our goal here is to cover the main aspects and see how SymPy performs some basic manipulations. Let us begin with some definitions.\nBrief mathematical background # General form # The most general form of a Fourier series is expressed as a sum of sine and cosine functions:\n$$ f(x) = \\frac{1}{2} a_0 + \\sum_n a_n \\cos(nx) + \\sum_n b_n \\sin(nx) $$where the coefficients, which derive from the orthonormal behavior of the sine and cosine functions, are\n$$ a_n = \\frac{1}{\\pi} \\int\\limits_{-\\pi}^{\\pi} f(x) \\cos(nx) dx \\\\ b_n = \\frac{1}{\\pi} \\int\\limits_{-\\pi}^{\\pi} f(x) \\sin(nx) dx $$Fourier series differ from Taylor series in several aspects. Here, two are worth highlighting:\nthe interval of convergence is always between \\(-\\pi\\) and \\(\\pi\\) in Fourier series, whereas in Taylor series the interval changes according to the function it is rare to find functions that cannot be expanded in a Fourier series, whereas there are more examples of functions that cannot be expanded by Taylor Changing the interval # In some contexts, such as quantum mechanics, it is useful to change the range of a Fourier series from \\((-\\pi, \\pi)\\) to \\((-L, L)\\). For that, the variable \\(x\\) must be replaced by \\(\\frac{\\pi x}{L}\\):\n$$ f(x) = \\frac{1}{2}a_0 + \\sum_n a_n \\cos \\frac{n \\pi x}{L} + \\sum_n b_n \\sin \\frac{n \\pi x}{L} $$where\n$$ a_n = \\frac{1}{L} \\int\\limits_{-L}^{L} f(x) \\cos \\frac{n \\pi x}{L} dx \\\\ b_n = \\frac{1}{L} \\int\\limits_{-L}^{L} f(x) \\sin \\frac{n \\pi x}{L} dx \\\\ $$ DFT # We have already seen here on the site that the sine and cosine functions are related to the exponential function through Euler\u0026rsquo;s formula. And we have also written about complex numbers. So, we can write another form of the Fourier series:\n$$ f(x) = \\sum\\limits_{-\\infty}^{\\infty} c_n \\exp \\left( \\frac{in \\pi x}{L} \\right) $$where\n$$ c_n = \\frac{1}{2L} \\int\\limits_{-L}^{L} f(x) \\exp \\left( -\\frac{in \\pi x}{L} \\right) $$In this case, the coefficients are complex and the series will represent only periodic functions. However, it is possible to modify the equation for nonperiodic functions. Consider \\(k = \\frac{n \\pi}{L}\\). Now, imagine \\(L \\to \\infty\\). As \\(L\\) increases, changes in \\(k\\) occur with smaller increments for each change in \\(n\\). That is, \\(\\Delta k = \\frac{\\pi}{L} \\Delta n \\). In the limit \\(\\Delta k \\to 0\\), \\(k\\) becomes a continuous variable, and the coefficients \\(c_k\\) can be described as continuous functions of \\(k\\), \\(c(k)\\). Therefore, substituting \\(\\Delta n = \\frac{L}{\\pi} \\Delta k\\):\n$$ f(x) = \\lim_{L \\to \\infty} \\sum_{-\\infty}^{\\infty} c_n \\exp \\left( \\frac{in \\pi x}{L} \\right) \\Delta n \\\\ f(x) = \\lim_{\\Delta k \\to 0} \\sum_{-\\infty}^{\\infty} \\frac{L c(k)}{\\pi} \\exp(ikx) \\Delta k \\\\ $$Now, this is the definition of an integral. Setting \\(\\frac{Lc(k)}{\\pi} = \\frac{g(k)}{\\sqrt{2\\pi}}\\), we have:\n$$ f(x) = \\frac{1}{\\sqrt{2\\pi}} \\int_{-\\infty}^{\\infty} g(k) \\exp(ikx) dk \\\\ g(k) = \\frac{1}{\\sqrt{2\\pi}} \\int_{-\\infty}^{\\infty} f(x) \\exp(-ikx) dx $$The equation for \\(f(x)\\) is called the Fourier integral, and \\(f(x)\\) and \\(g(k)\\) are called continuous Fourier transforms of each other. For computational calculations, it is sometimes more convenient to work with discrete samples, which gives rise to the discrete Fourier transforms (DFT).\nFor example, suppose a function of time, such as sine or cosine, expressed as \\(\\sin(2\\pi \\nu t)\\) or \\(\\cos(2\\pi \\nu t)\\). Since it is a periodic function, we need samples for only one period \\(\\tau_0\\). The DFT function is defined by\n$$ F(\\nu) = \\frac{1}{N} \\sum_{k} f(k) \\exp(2\\pi i \\nu k) $$where \\(N\\) is the number of samples taken from the period \\(\\tau_0\\) and \\(k\\) ranges from \\(-t\\) to \\(t-1\\) in integers centered at zero. The Fourier transform is then said to transform the function from the time domain to the frequency domain. Considering Euler\u0026rsquo;s formula \\(e^{i \\theta} = \\cos \\theta + i \\sin \\theta\\):\n$$ F(\\nu) = \\frac{1}{N} \\sum_{k} f(k) (\\cos(2 \\pi \\nu k) + i \\sin(2\\pi \\nu k)) $$Obviously, more samples lead to better outputs. Nowadays there is the fast Fourier transform algorithm, which requires the number of points to be a power of two.\nIt is convenient to separate the real part from the imaginary part in the previous equation:\n$$ \\operatorname{Re}(\\nu) = \\frac{1}{N} \\sum_{k} f(k) \\cos(2 \\pi \\nu k) \\\\ i \\operatorname{Im}(\\nu) = \\frac{i}{N} \\sum_{k} f(k) \\sin(2 \\pi \\nu k) $$The absolute value of \\(F(\\nu)\\) is given by\n$$ |F(\\nu)|^2 = F^*(\\nu) F(\\nu) = (\\operatorname{Re}(\\nu))^2 + (\\operatorname{Im}(\\nu))^2 $$A plot of \\(|F(\\nu)|^2\\) versus \\(\\nu\\) is called the power spectrum of the Fourier transform.\nDifference between Fourier series and Fourier transform # Note in the previous subsection that the idea of the Fourier transform arose when a way to obtain series for nonperiodic functions was sought. That is the key point of the distinction.\nA Fourier series is used to represent a periodic function through a discrete sum of complex exponentials. A Fourier transform is used to represent a general, nonperiodic function through a continuous superposition or integral of complex exponentials. A Fourier transform can be seen as the limit of the Fourier series of a function as the period approaches infinity, so that the limits of integration change from one period to \\((-\\infty, \\infty)\\).\nParity of functions # Finally, we saw with Taylor that even or odd functions are convenient because they facilitate some mathematical manipulations. The same occurs in Fourier series.\nFor odd functions, the coefficients \\(a\\) are zero and the series becomes\n$$ f(x) = \\sum_{n=1}^{\\infty} b_n \\sin(nx) $$where\n$$ b_n = \\frac{2}{\\pi} \\int_0^{\\pi} f(x) \\sin(nx) dx $$For even functions, the coefficients \\(b\\) are zero and the series becomes\n$$ f(x) = \\frac{1}{2}a_0 + \\sum_{n=1}^{\\infty} a_n \\cos(nx) $$where\n$$ a_n = \\frac{2}{\\pi} \\int_0^{\\pi} f(x) \\cos(nx) dx $$That was a very brief review of the subject. It will certainly appear more times here on the site, so many applications will be seen. Now, let us see how SymPy performs Fourier operations.\nFourier series in SymPy # We can create a Fourier series for a function with fourier_series:\nfrom sympy import fourier_series fourier = fourier_series(f(x)) fourier $$\\displaystyle \\frac{\\sin{\\left(x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(x \\right)} dx}{\\pi} + \\frac{\\sin{\\left(2 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(2 x \\right)} dx}{\\pi} + \\frac{\\cos{\\left(x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(x \\right)} dx}{\\pi} + \\frac{\\cos{\\left(2 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(2 x \\right)} dx}{\\pi} + \\frac{\\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} dx}{2 \\pi} + \\ldots$$Note that it was given in the general form, making it clear that it is a sequence of sums.\nWe can obtain each of the coefficients of the general form:\nfourier.a0 \\(\\displaystyle \\frac{\\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} dx}{2 \\pi}\\)\nfourier.an $$\\displaystyle \\left[\\frac{\\cos{\\left(x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(x \\right)} dx}{\\pi}, \\frac{\\cos{\\left(2 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(2 x \\right)} dx}{\\pi}, \\frac{\\cos{\\left(3 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(3 x \\right)} dx}{\\pi}, \\frac{\\cos{\\left(4 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\cos{\\left(4 x \\right)} dx}{\\pi}, \\ldots\\right]$$fourier.bn $$\\displaystyle \\left[\\frac{\\sin{\\left(x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(x \\right)} dx}{\\pi}, \\frac{\\sin{\\left(2 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(2 x \\right)} dx}{\\pi}, \\frac{\\sin{\\left(3 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(3 x \\right)} dx}{\\pi}, \\frac{\\sin{\\left(4 x \\right)} \\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} \\sin{\\left(4 x \\right)} dx}{\\pi}, \\ldots\\right]$$We saw in the mathematical background of the previous section that it is common to denote the limit of the interval by \\(L\\). We can, then, check the limit of the interval:\nfourier.L \\(\\displaystyle \\pi\\)\nAlso in the previous section, we saw that we can write it in exponential form. To do this, we use the rewrite method, requesting that the exp function be used:\nfourier.rewrite(exp) $$\\displaystyle \\frac{\\left(\\frac{e^{i x}}{2} + \\frac{e^{- i x}}{2}\\right) \\int\\limits_{- \\pi}^{\\pi} \\left(\\frac{e^{i x}}{2} + \\frac{e^{- i x}}{2}\\right) f{\\left(x \\right)} dx}{\\pi} - \\frac{i \\left(e^{i x} - e^{- i x}\\right) \\int\\limits_{- \\pi}^{\\pi} \\left(- \\frac{i \\left(e^{i x} - e^{- i x}\\right) f{\\left(x \\right)}}{2}\\right) dx}{2 \\pi} + \\frac{\\left(\\frac{e^{2 i x}}{2} + \\frac{e^{- 2 i x}}{2}\\right) \\int\\limits_{- \\pi}^{\\pi} \\left(\\frac{e^{2 i x}}{2} + \\frac{e^{- 2 i x}}{2}\\right) f{\\left(x \\right)} dx}{\\pi} - \\frac{i \\left(e^{2 i x} - e^{- 2 i x}\\right) \\int\\limits_{- \\pi}^{\\pi} \\left(- \\frac{i \\left(e^{2 i x} - e^{- 2 i x}\\right) f{\\left(x \\right)}}{2}\\right) dx}{2 \\pi} + \\frac{\\int\\limits_{- \\pi}^{\\pi} f{\\left(x \\right)} dx}{2 \\pi} + \\ldots$$Let us look at some more practical examples.\nPlots and function parity # Let us take a very simple function as an example: \\(f(x) = x\\). First, let us create the Fourier series for this function:\nfrom sympy import pi s = fourier_series(x, (x, -pi, pi)) s \\(\\displaystyle 2 \\sin{\\left(x \\right)} - \\sin{\\left(2 x \\right)} + \\frac{2 \\sin{\\left(3 x \\right)}}{3} + \\ldots\\)\nAs for parity, this function is odd, since \\(f(-x) = - f(x)\\). From what we saw earlier, the coefficients a should be zero:\ns.a0, s.an, s.bn \\(\\displaystyle \\left( 0, \\ \\left[0, 0, 0, 0, \\ldots\\right], \\ \\left[2 \\sin{\\left(x \\right)}, - \\sin{\\left(2 x \\right)}, \\frac{2 \\sin{\\left(3 x \\right)}}{3}, - \\frac{\\sin{\\left(4 x \\right)}}{2}, \\ldots\\right]\\right)\\)\nWith the truncate method we can define how many terms we want in our series. For example, with five terms:\ns.truncate(5) \\(\\displaystyle 2 \\sin{\\left(x \\right)} - \\sin{\\left(2 x \\right)} + \\frac{2 \\sin{\\left(3 x \\right)}}{3} - \\frac{\\sin{\\left(4 x \\right)}}{2} + \\frac{2 \\sin{\\left(5 x \\right)}}{5}\\)\nThe truncate, as used above, shows only nonzero terms. If we pass None to the method, it returns a generator:\ngen_fourier = s.truncate(None) gen_fourier \u0026lt;generator object SeriesBase.__iter__ at 0x74d969cd09e0\u0026gt; We can work with this generator using islice, as we did earlier with Taylor series. For example, we can generate the first 10 terms and store them in a tuple:\ntuple(islice(gen_fourier, 10)) $$\\displaystyle \\left( 0, \\ 2 \\sin{\\left(x \\right)}, \\ - \\sin{\\left(2 x \\right)}, \\ \\frac{2 \\sin{\\left(3 x \\right)}}{3}, \\ - \\frac{\\sin{\\left(4 x \\right)}}{2}, \\ \\frac{2 \\sin{\\left(5 x \\right)}}{5}, \\ - \\frac{\\sin{\\left(6 x \\right)}}{3}, \\ \\frac{2 \\sin{\\left(7 x \\right)}}{7}, \\ - \\frac{\\sin{\\left(8 x \\right)}}{4}, \\ \\frac{2 \\sin{\\left(9 x \\right)}}{9}\\right)$$Note that the first term is zero, referring to the \\(a_0\\) of the general form of a Fourier series.\nAnd what is the effect of the number of terms? In the plot below, we see that the more terms, the better the approximation of our function:\nfig, ax = plt.subplots(figsize=(10, 10)) axins = zoomed_inset_axes(ax, 2, loc=\u0026#39;lower right\u0026#39;) # zoom = 2 p_x = plot(x, (x, -3, 3), show=False, label=r\u0026#39;\\\\(f(x) = x\\\\)\u0026#39;) move_sympy_plot_to_axes(p_x, ax) data_x = p_x[0].get_points() axins.plot(*data_x) for n in range(1, 16, 2): fourier = s.truncate(n) p_fourier = plot(fourier, (x, -3, 3), show=False, label=f\u0026#34;Fourier - n = {n}\u0026#34;) move_sympy_plot_to_axes(p_fourier, ax) data_fourier = p_fourier[0].get_points() axins.plot(*data_fourier) x1, y1 = -0.5, -0.5 x2, y2 = 0.5, 0.5 axins.set_xlim(x1, x2) axins.set_ylim(y1, y2) mark_inset(ax, axins, loc1=1, loc2=2, fc=\u0026#34;none\u0026#34;, ec=\u0026#34;0.6\u0026#34;) ax.legend() plt.show() In case it is still not clear, see the following animation:\nThe animation code is in this repository:\nchicolucio/fourier Fourier series demonstration Python 0 0 Application in electrical engineering # To make the practical importance of the subject more evident, let us look at a real application.\nA rectifier is an electrical device that converts alternating current into direct current. It has several applications, for example, in power supplies for electronic devices. Let us look at an example of a full-wave rectifier. In this type of rectifier, the input wave is converted into a wave of constant polarity (positive or negative). Mathematically, this corresponds to the absolute value of the function.\nLet us create our symbols and the expression for the input wave function:\nfrom sympy import Abs amplitude = Symbol(\u0026#39;A\u0026#39;, positive=True, real=True) period = Symbol(\u0026#39;P\u0026#39;, positive=True, real=True) x = Symbol(\u0026#39;x\u0026#39;, real=True) expr_input = amplitude * sin((2 * pi / period) * x) expr_input \\(\\displaystyle A \\sin{\\left(\\frac{2 \\pi x}{P} \\right)}\\)\nLet us look at the plot of this wave. As an example, let us consider amplitude and period with value 1:\nfrom sympy.plotting import plot plot(expr_input.subs({period: 1, amplitude: 1}), (x, -2, 2)); See what an alternating wave looks like. As written earlier, the rectification process is mathematically equivalent to the absolute value of the function. So, let us obtain the expression using SymPy\u0026rsquo;s Abs function:\nexpr_output = Abs(expr_input) expr_output \\(\\displaystyle A \\left|{\\sin{\\left(\\frac{2 \\pi x}{P} \\right)}}\\right|\\)\nWe can plot this new function, with the same period and amplitude values (which refer to the input function):\nplot(expr_output.subs({period: 1, amplitude: 1}), (x, -2, 2)); Note that in the plot of the rectified wave there is only the positive part and the period was halved (it went from 1 to 0.5). Thus, the output frequency is twice the input frequency. In case you did not understand, see the figure with both functions overlaid:\np_input = plot(expr_input.subs({period: 1, amplitude: 1}), (x, -2, 2), show=False, label=\u0026#39;Input\u0026#39;) p_output = plot(expr_output.subs({period: 1, amplitude: 1}), (x, -2, 2), show=False, label=\u0026#39;Output\u0026#39;) fig, ax = plt.subplots(figsize=(12, 6)) move_sympy_plot_to_axes(p_input, ax) move_sympy_plot_to_axes(p_output, ax) ax.lines[0].set(linestyle=\u0026#39;:\u0026#39;, linewidth=3) ax.legend() plt.show() Now, let us obtain our Fourier series (frequency domain) for the rectified wave function (time domain):\ns = fourier_series(expr_output, (x, 0, period)) s \\(\\displaystyle - \\frac{4 A \\cos{\\left(\\frac{4 \\pi x}{P} \\right)}}{3 \\pi} - \\frac{4 A \\cos{\\left(\\frac{8 \\pi x}{P} \\right)}}{15 \\pi} + \\frac{2 A}{\\pi} + \\ldots\\)\nLet us explore the coefficients:\ns.a0 \\(\\displaystyle \\frac{2 A}{\\pi}\\)\ns.an \\(\\displaystyle \\left[0, - \\frac{4 A \\cos{\\left(\\frac{4 \\pi x}{P} \\right)}}{3 \\pi}, 0, - \\frac{4 A \\cos{\\left(\\frac{8 \\pi x}{P} \\right)}}{15 \\pi}, \\ldots\\right]\\)\ns.bn \\(\\displaystyle \\left[0, 0, 0, 0, \\ldots\\right]\\)\nNote how the coefficients \\(b_n\\) are zero. As we saw earlier, this is characteristic of even functions. The plot confirms that it is an even function, since it is symmetric with respect to the y axis.\nLet us substitute values into the function. Just as an example, let us consider amplitude and period with value 1, as done before:\ns.subs({period: 1, amplitude: 1}).an \\(\\displaystyle \\left[0, - \\frac{4 \\cos{\\left(4 \\pi x \\right)}}{3 \\pi}, 0, - \\frac{4 \\cos{\\left(8 \\pi x \\right)}}{15 \\pi}, \\ldots\\right]\\)\nSee how there is an alternation with zero terms.\nNow, let us look at the plot of this series. Since it is a series, we need to define how many terms we will add. Let us start with a low value, such as 5:\nplot(s.truncate(5).subs({period: 1, amplitude: 1}), (x, -2, 2)); How strange. Our series does resemble the function, but there is a negative part in the plot. Let us plot with more terms. For example, with 10 terms:\nplot(s.truncate(10).subs({period: 1, amplitude: 1}), (x, -2, 2)); Now we have a better approximation of our function. If you look at the plot, you will see that the minima are not touching the horizontal axis, which would be expected for the given values. If we add more terms, we will have an even better approximation. Let us see with 20 terms:\nplot(s.truncate(20).subs({period: 1, amplitude: 1}), (x, -2, 2)); Now the minima are closer to zero. We could keep adding more terms until some approximation criterion was satisfied.\nFourier transforms in SymPy # To finish the subject, let us look at the fourier_transform function. Let us start by creating a function:\nexpr = exp(-x**2) expr \\(\\displaystyle e^{- x^{2}}\\)\nThe function created is a very simple Gaussian. Gaussian functions are widely studied because of their applications in statistics and signal processing. Let us look at its plot:\nplot(expr, (x, -5, 5)); Now, let us do the Fourier transform using the notation shown in the theoretical part of the article:\nfrom sympy import fourier_transform k = Symbol(\u0026#39;k\u0026#39;, real=True) ft = fourier_transform(expr, x, k) ft \\(\\displaystyle \\sqrt{\\pi} e^{- \\pi^{2} k^{2}}\\)\nSee that using the fourier_transform function is very simple. The function we got as output is also a Gaussian. In fact, this is a property of Gaussians. Let us look at the plot of the output function of the transform:\nplot(ft, (k, -5, 5)); Note that it is a Gaussian with a smaller width than the input Gaussian (before the transform). Let us overlay the plots to make this more evident:\np_input = plot(expr, (x, -5, 5), show=False, label=\u0026#39;Input\u0026#39;) p_output = plot(ft, (k, -5, 5), show=False, label=\u0026#39;Output\u0026#39;) fig, ax = plt.subplots(figsize=(12, 6)) move_sympy_plot_to_axes(p_input, ax) move_sympy_plot_to_axes(p_output, ax) ax.lines[0].set(linestyle=\u0026#39;:\u0026#39;, linewidth=3) ax.set_xlabel(\u0026#39;\u0026#39;) ax.set_ylabel(\u0026#39;\u0026#39;) ax.legend() plt.show() And this is a general property of functions and their Fourier transforms: if a function is more \u0026ldquo;spread out\u0026rdquo; in x, its Fourier transform will be more \u0026ldquo;concentrated\u0026rdquo; in k. And vice versa. It is not possible to arbitrarily compact both the function and its transform. This is a very informal way of stating an uncertainty principle.\nIt is quite common to begin the study of Fourier transforms with audio signals, hence the terminology we have already used in this article of saying that the input function is in the time domain and the output one in the frequency domain. Thus, in signal processing, it is said that it is not possible to precisely localize a signal (input function) simultaneously in the time domain and in the frequency domain (output function, Fourier transform).\nHowever, nothing prevents it from being another context. For example, in quantum mechanics the momentum and position wave functions are Fourier transform pairs within a margin of error. Hence the famous Heisenberg uncertainty principle. Not by chance, several Gaussian functions appear during the study of quantum mechanics. In fact, Gaussians are very common in many contexts.\nConclusion # Did you imagine it would be possible, in a single text, to go from Greek philosophy to quantum mechanics? And using programming? This article taught me a lot while I wrote it and, I hope, it teaches you a lot too, reader. As important as knowing how to do mathematical operations in a language such as Python is knowing the theoretical background to see the many possible applications. And a bit of history helps, doesn\u0026rsquo;t it :-)\nSee you next time.\n","date":"2022-05-05","externalUrl":null,"permalink":"/2022/05/infinite-series-calculus-python/","section":"Blog","summary":"A good part of the study of calculus is devoted to what are called infinite series. But what, exactly, are infinite series? And how are they related to the paradox of Achilles and the tortoise, one of the so-called paradoxes of Zeno of Elea, the Greek philosopher? What is the connection with quantum mechanics and the uncertainty principle? And where does Python come in? That is what we will find out in this article.","title":"Infinite Series - Calculus with Python and SymPy","type":"posts"},{"content":"A series of articles on SymPy, a Python library for symbolic mathematics.\nThe articles are listed below, in the order they were written.\n","date":"2022-05-05","externalUrl":null,"permalink":"/series/sympy/","section":"Series","summary":"","title":"SymPy","type":"series"},{"content":"","date":"2022-04-20","externalUrl":null,"permalink":"/tags/balanco-de-massa/","section":"tags","summary":"","title":"balanço de massa","type":"tags"},{"content":"","date":"2022-04-20","externalUrl":null,"permalink":"/tags/chempy/","section":"tags","summary":"","title":"ChemPy","type":"tags"},{"content":"Quality control analyst and data analyst. Undergraduate student in Industrial Chemistry at the Federal University of Rio de Janeiro (UFRJ) and Chemistry Technician from the Federal Institute of Rio de Janeiro (IFRJ). Beyond Chemistry, I have a strong interest and experience in the Python programming language, where my main skills are with scientific tools.\nFollow me on LinkedIn for more content.\n","date":"2022-04-20","externalUrl":null,"permalink":"/authors/helenabenevenuto/","section":"Authors","summary":"","title":"Helena Benevenuto","type":"authors"},{"content":"","date":"2022-04-20","externalUrl":null,"permalink":"/tags/mass-balance/","section":"tags","summary":"","title":"mass balance","type":"tags"},{"content":" Mass balance equations are important for practically all calculations in solving chemical engineering problems. For this reason, they will be present throughout your undergraduate course and professional career. Mass balances are carried out during the operation of processing plants over different periods of time, with the goal of controlling processes, maintaining production, and increasing efficiency.\nIn this article, we will solve two mass balance problems using the SymPy library and review some important principles.\nThe Concept of Mass Balance # Mass balance is the application of the Law of Conservation of Mass:\n“Matter is neither created nor destroyed.” Therefore, when the process is continuous and operates at steady state, the accumulation inside a piece of equipment is zero:\nEverything that enters must leave. Remember, a continuous process is one in which mass passes continuously across the system boundaries. A steady-state process, in turn, is one in which the values of the process variables do not change with time.\nWith a relatively simple equation we can represent, in general form, the balance of any quantity:\nIn terms of a balance of a quantity across the system boundaries:\nAnalyzing a real case # Let\u0026rsquo;s analyze a simple problem. Suppose that 100 mol/h of a mixture with 30% of component A and 70% of component B is separated by distillation into two fractions. The top stream contains 70% of component A and the bottom stream has 20% of component B. Let\u0026rsquo;s calculate the molar flow rate of each stream.\nThe first step in solving a molar balance problem is to draw the process diagram, specifying the boundaries and adding the known variables:\nAfter analyzing the figure, we see that there is crossing of the process boundaries, so this problem operates as a continuous process. Also, the values of the process variables do not change with time, so it is at steady state. Furthermore, we note that no chemical reaction takes place.\nSolving with Python and SymPy # To solve the problem, let\u0026rsquo;s import the libraries we will use:\nfrom collections import namedtuple import sympy sympy.init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) Let\u0026rsquo;s start by creating the variables with the information from the problem statement and completing them with the mole fractions that were not given, but that are easily determined from the following constraints:\n$$ \\begin{aligned} x_a+x_b=1 \\Rightarrow 0.7+x_b=1 \\Rightarrow x_b=0.3 \\\\ y_a+y_b=1 \\Rightarrow 0.2+y_b=1 \\Rightarrow y_b=0.8 \\end{aligned} $$Where xa is the mole fraction of component A and xb that of component B, both in the top stream. ya is the mole fraction of component A and yb that of component B, both in the bottom stream.\nfeed_rate = 100 # molar flow rate in mol/h Composition = namedtuple(\u0026#39;Composition\u0026#39;, \u0026#39;A B\u0026#39;) feed = Composition(0.3, 0.7) # feed composition stream_1 = Composition(0.7, 0.3) # top stream composition stream_2 = Composition(0.2, 0.8) # bottom stream composition Note that I used the namedtuple function. It lets us access values using field names instead of the position indices of a tuple, which makes the code more readable. In case that is not clear yet, we will see it applied right below.\nNow, let\u0026rsquo;s define the molar flow rate symbols using LaTeX. We usually put a dot above the flow symbol to denote a rate. Thus, the molar flow rate (\\(\\dot{n}\\)) of a process stream is the number of moles (\\(n\\)) transported per unit of time.\n$$ \\dot{n} = \\frac{n}{t} $$flow_rate_1, flow_rate_2 = sympy.symbols(\u0026#39;\\dot{n_1} \\dot{n_2}\u0026#39;) flow_rate_1, flow_rate_2 \\( (\\dot{n_1}, \\ \\dot{n_2}) \\)\nThe next step is to write the equations we need to solve in terms of the known variables and the unknowns. Since there is no chemical reaction in this problem and it involves a molar balance at steady state, we have:\ntotal molar flow rate in = total molar flow rate out This is consistent with what we wrote at the beginning of the article about the accumulation term inside a piece of equipment being zero:\nEverything that enters must leave. Therefore, for our exercise, the overall molar balance is:\n$$\\dot{n_1}+\\dot{n_2}=100$$And the component molar balance is:\n$$ \\begin{aligned} 0.7\\cdot\\dot{n_1}+0.2\\cdot\\dot{n_2}=0.3\\cdot100 \\quad \\text{ (component A)} \\\\ 0.3\\cdot\\dot{n_1}+0.8\\cdot\\dot{n_2}=0.7\\cdot100 \\quad \\text{ (component B)} \\end{aligned} $$We need to pass this information to SymPy by creating a system of equations. We have already written about this in this article; let\u0026rsquo;s apply it to our exercise:\nsystem = (sympy.Eq(flow_rate_1 + flow_rate_2, feed_rate), # overall sympy.Eq(stream_1.A * flow_rate_1 + stream_2.A * flow_rate_2, feed.A * feed_rate), # component A sympy.Eq(stream_1.B * flow_rate_1 + stream_2.B * flow_rate_2, feed.B * feed_rate) # component B ) # showing each equation of the system for equation in system: display(equation) $$ \\begin{aligned} \\dot{n_1}+\\dot{n_2}\u0026=100 \\\\ 0.7\\dot{n_1}+0.2\\dot{n_2}\u0026=30.0 \\\\ 0.3\\dot{n_1}+0.8\\dot{n_2}\u0026=70.0 \\end{aligned} $$In Python, the symbols = and == are the assignment and equality operators, respectively, and we cannot use them to create equalities in equations. To set up an equation, SymPy has the Eq function, which creates a symbolic equality.\nAnother important point is that we can see the advantage of using the namedtuple function when we access the components. We named the fields A and B and accessed the values using dot notation, which improves the readability of the code.\nWith the system created, we just need to ask for it to be solved with SymPy\u0026rsquo;s solve function:\nsolution = sympy.solve(system) solution \\( \\{ \\dot{n_1}: 20.0, \\ \\dot{n_2}: 80.0 \\} \\)\nSo, the top stream has a flow rate of 20 mol/h and the bottom stream has a flow rate of 80 mol/h.\nThis line of reasoning applies to any number of components. As long as the number of independent equations is equal to the number of unknowns, the system can be solved. Let\u0026rsquo;s look at one more example, this time with 3 outlet streams and 3 components.\nA real case with more components # Suppose that 100 mol/h of a mixture with 60% of component A, 25% of component B, and 15% of component C is separated by distillation into three fractions. The streams contain:\ni) 80% of component A and 16% of component B; ii) 50% of component A and 23.3% of component B; iii) 25% of component A and 50% of component B.\nLet\u0026rsquo;s calculate the molar flow rate of each stream. To do so, we start by drawing the process diagram, specifying the boundaries and adding the known variables:\nSolving with Python and SymPy # We need to create the variables with the information needed to solve the problem. Let\u0026rsquo;s extract this data from the problem statement, and the remaining mole fractions can be determined from the following constraints:\n$$ \\begin{aligned} x_a+x_b+x_c=1 \u0026 \\Rightarrow 0.8+0.16+x_c=1 \u0026\u0026 \\Rightarrow x_c=0.04 \\\\ y_a+y_b+y_c=1 \u0026 \\Rightarrow 0.5+0.233+y_c=1 \u0026\u0026 \\Rightarrow y_c=0.267 \\\\ z_a+z_b+z_c=1 \u0026 \\Rightarrow 0.25+0.5+z_c=1 \u0026\u0026 \\Rightarrow z_c=0.25 \\end{aligned} $$feed_rate = 100 # molar flow rate in mol/h Composition = namedtuple(\u0026#39;Composition\u0026#39;, \u0026#39;A B C\u0026#39;) feed = Composition(0.6, 0.25, 0.15) # feed composition stream_1 = Composition(0.8, 0.16, 0.04) # top stream composition stream_2 = Composition(0.5, 0.233, 0.267) # middle stream composition stream_3 = Composition(0.25, 0.5, 0.25) # bottom stream composition Now, let\u0026rsquo;s define the molar flow rate symbols using LaTeX:\nflow_rate_1, flow_rate_2, flow_rate_3 = sympy.symbols(\u0026#39;\\dot{n_1} \\dot{n_2} \\dot{n_3}\u0026#39;) flow_rate_1, flow_rate_2, flow_rate_3 \\( \\dot{n_1}, \\ \\dot{n_2}, \\ \\dot{n_3} \\)\nIn this problem:\ntotal molar flow rate in = total molar flow rate out Based on that, let\u0026rsquo;s write the equations we need to solve. The line of reasoning is the same as in the previous case; only the number of variables and equations grows.\nThe overall molar balance is:\n$$\\dot{n_1}+\\dot{n_2}+\\dot{n_3}=100$$And the component molar balance is:\n$$ \\begin{aligned} 0.8\\cdot\\dot{n_1}+0.5\\cdot\\dot{n_2}+0.25\\cdot\\dot{n_3}=0.6\\cdot100 \\quad \\text{ (component A)} \\\\ 0.16\\cdot\\dot{n_1}+0.233\\cdot\\dot{n_2}+0.5\\cdot\\dot{n_3}=0.25\\cdot100 \\quad \\text{ (component B)} \\\\ 0.04\\cdot\\dot{n_1}+0.267\\cdot\\dot{n_2}+0.25\\cdot\\dot{n_3}=0.15\\cdot100 \\quad \\text{ (component C)} \\end{aligned} $$Now let\u0026rsquo;s pass this information to SymPy by creating a system of equations:\nsystem = (sympy.Eq(flow_rate_1 + flow_rate_2 + flow_rate_3, feed_rate), sympy.Eq(stream_1.A * flow_rate_1 + stream_2.A * flow_rate_2 + stream_3.A * flow_rate_3, feed.A * feed_rate), sympy.Eq(stream_1.B * flow_rate_1 + stream_2.B * flow_rate_2 + stream_3.B * flow_rate_3, feed.B * feed_rate), sympy.Eq(stream_1.C * flow_rate_1 + stream_2.C * flow_rate_2 + stream_3.C * flow_rate_3, feed.C * feed_rate)) # showing each equation of the system for equation in system: display(equation) $$ \\begin{aligned} \\dot{n_1}+\\dot{n_2}+\\dot{n_3}\u0026=100 \\\\ 0.8\\dot{n_1}+0.5\\dot{n_2}+0.25\\dot{n_3}\u0026=60.0 \\\\ 0.16\\dot{n_1}+0.233\\dot{n_2}+0.5\\dot{n_3}\u0026=25.0 \\\\ 0.04\\dot{n_1}+0.267\\dot{n_2}+0.25\\dot{n_3}\u0026=15.0 \\end{aligned} $$After creating the system, we need to ask for it to be solved with SymPy\u0026rsquo;s solve function:\nsolution = sympy.solve(system) solution \\( \\{ \\dot{n_1}: 50.0404203718674, \\ \\dot{n_2}: 29.9110751818917, \\ \\dot{n_3}: 20.0485044462409 \\} \\)\nThe top stream has a flow rate of 50 mol/h, the middle stream 30 mol/h, and the bottom stream 20 mol/h.\nConclusion # Solving mass balance equations got simpler with Python, didn\u0026rsquo;t it?! In this article, we saw how some features of the SymPy library automate the solution of mass balance problems. Moreover, we can easily apply the tools used here to solve other types of balances and chemical engineering problems.\nThis article is a collaboration between me, Helena Benevenuto, and Ciência Programada. Follow me on LinkedIn for more chemistry content.\nShare this article on your networks and follow the Ciência Programada project to stay up to date. See you next time.\n","date":"2022-04-20","externalUrl":null,"permalink":"/2022/04/mass-balance-python-sympy/","section":"Blog","summary":"Mass balance equations are important for practically all calculations in solving chemical engineering problems. In this article, we will solve two mass balance problems using the SymPy library and review some important principles.","title":"Mass Balance with Python and SymPy","type":"posts"},{"content":"","date":"2022-04-15","externalUrl":null,"permalink":"/tags/drops/","section":"tags","summary":"","title":"drops","type":"tags"},{"content":" We have already written here on the site about the concepts of limit and derivative and how we can do the related computations with Python. Continuing our journey through calculus, in this article we will see how to work with integrals using SymPy and SciPy, Python packages. Along the way, we will pick up some notions of numerical methods.\nThere are two approaches to integral calculus. One considers the integral as an antiderivative. That is, integration is regarded as the inverse of differentiation. The other considers the integral as the sum of many infinitesimal elements. This second approach is the one that allows us to give physical meaning to an integral. Let\u0026rsquo;s look at a bit of each.\nIntegral as an antiderivative # In calculus courses, we usually represent the differentiation of a function \\(y = f(x)\\) as:\n$$ \\frac{dy}{dx} = \\frac{df(x)}{dx} = f'(x) $$Which we can also represent in differential form\n$$ dy = f'(x)dx $$We have already seen how to handle derivatives in SymPy in this article.\nIn our context, we can ask ourselves the following question: which function \\(F(x)\\), when differentiated, results in the function \\(f^{\\prime}(x)\\)? Such a function is called the integral of the differential and is written as\n$$ F(x) = \\int f'(x) dx $$For example, for \\(f^{\\prime}(x) = 2x\\), the function we are looking for is\n$$ F(x) = \\int 2xdx = x^2 + C $$Note the constant of integration C. Let whoever has never forgotten to write the constant of integration on a calculus exam cast the first derivative! Given the differentiation rules studied in any calculus course, the derivative of a constant is zero, so there are infinitely many functions that satisfy our conditions. Just as an example, in the following plot we see 5 of the antiderivatives of \\(2x\\).\nThe code for this plot is in this repository of mine.\nIntegral as a sum of infinitesimal elements # The previous approach is the mathematical one, the one that lets us know how to obtain an expression for an integral and that gives rise to the rules we learn in calculus courses. But what is the meaning of an integral?\nConsider that our function of interest is\n$$ f(x) = \\frac{1}{x^2 + 1} $$From any table of integrals, you will see that\n$$ \\int \\frac{1}{x^2 + 1} dx = \\arctan(x) + C $$And, with a bit of the SymPy we saw in the article on plotting, we can obtain the plot of the function over a given domain:\nfrom sympy import Symbol, plot x = Symbol(\u0026#39;x\u0026#39;) expr = 1 / (x**2 + 1) plot(expr, (x, -0.5, 5.5)); The integral we obtained is called indefinite, because we did not specify the limits of integration. If the limits of integration are specified, the integral is called definite and we can obtain a value. For example,\n$$ \\int_0^5 \\frac{1}{x^2 + 1} dx = \\arctan(5) - \\arctan(0) = \\arctan(5) \\approx 1.3734 $$We could highlight this region between the limits of integration in the plot by shading it:\nAnd\u0026hellip; that is the meaning of an integral, more specifically of a definite integral! It is the area of the region between the graph and the horizontal axis of the plane.\nNow that we have a graphical meaning, it is easy to extend it to a physical meaning. For example, if the vertical axis is the pressure on a gas and the horizontal axis is the change in the gas volume, the area is simply the work done on the gas. That is why physics and engineering courses take such an interest in areas under graphs.\nOK, when we have the plot it is relatively easy to see the meaning. But how do we obtain a value? After all, for simple geometric figures, such as rectangles, triangles and circles, we learn formulas to compute the area in basic math courses. But what about the area of a region like the one in the plot above? How do a calculator or a computer manage to present a value to you?\nWell, we know how to compute the area of, say, a rectangle. So we keep placing rectangles side by side, trying to imitate the shape of the graph as closely as possible. Yes, seriously. That is the idea behind the so-called Riemann sum. See in the following animation how, by using ever smaller rectangles, we get an ever better approximation of our integral:\nWe see three plots in the figure. When we decide the number of subintervals into which we will divide our interval, we can align the rectangles at the left or right endpoint of each subinterval, or at the midpoint of the interval. That is why there are three plots.\nDid you like the animation? Here is the repository with the code:\nchicolucio/integrals Computing integrals Python 5 0 In other words, the values that calculators or computer programs return for integrals are the results of numerical approximations. Following this line of reasoning, there are other approximation methods, such as the trapezoid rule or Simpson\u0026rsquo;s rules. These numerical approximations are usually covered in Numerical Analysis courses.\nOK, and Python?\nIntegrals with SymPy # Let\u0026rsquo;s create this same function we used as an example with SymPy, taking the opportunity to import other functions we will use later:\n# setup for nicer outputs in the article, can be ignored from sympy import init_printing init_printing(scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) from sympy import integrate, Symbol, dsolve, Function, diff, Eq x = Symbol(\u0026#39;x\u0026#39;) expr = 1 / (x**2 + 1) expr \\(\\displaystyle \\frac{1}{x^{2} + 1}\\)\nTo obtain the indefinite integral of the function, just pass its expression expr to the integrate function:\nintegrate(expr, x) \\(\\displaystyle \\operatorname{atan}{\\left(x \\right)}\\)\nSame result that appeared in the text earlier except for one detail\u0026hellip; where is the constant of integration?\nSymPy \u0026ldquo;simplifies\u0026rdquo; the result by omitting the constant of integration, in order to make it easier to pass the result to other SymPy functions if desired. However, we must be aware that it exists. Or we can force it to show up :-)\nWe saw earlier that we can interpret an integral as an antiderivative, which is nothing more than a function that equals a given derivative. So we have a differential equation, and we have already seen how to solve differential equations with SymPy.\nLet\u0026rsquo;s create the function we are looking for and set up our equation:\nF = Function(\u0026#39;F\u0026#39;) diff_eq = Eq(diff(F(x)), expr) diff_eq \\(\\displaystyle \\frac{d}{d x} F{\\left(x \\right)} = \\frac{1}{x^{2} + 1}\\)\nTherefore, the function we are looking for (which is the desired integral) is:\ndsolve(diff_eq, F(x)) \\(\\displaystyle F{\\left(x \\right)} = C_{1} + \\operatorname{atan}{\\left(x \\right)}\\)\nThere is the constant of integration ;-)\nSee how important it is to know the fundamentals of what you are doing. Unfortunately, many teachers are wary of introducing computational tools in teaching, thinking that students will use them and not learn to do the math \u0026ldquo;by hand\u0026rdquo;. Likewise, many students think it is a waste of time to manually do what a computer does in a matter of seconds. The point is that the computer only does what it is asked to do. If you don\u0026rsquo;t know what to ask for\u0026hellip;\nBesides, critically analyzing the result is up to the user. The absence of the constant of integration in the return of integrate was a choice of the developers, but noticing that it was missing must come from the user, based on their knowledge of calculus. It is to be expected that everything presented here in terms of calculus is known to anyone who has taken a basic calculus course.\nSo, with this very simple example involving a simple constant of integration, I hope that both teachers and students are aware that basic knowledge of the subject is still necessary when using programming.\nContinuing\u0026hellip;\nAnd definite integrals? Simple, we pass a second argument to integrate with the variable symbol and the desired interval:\nintegrate(expr, (x, 0, 5)) \\(\\displaystyle \\operatorname{atan}{\\left(5 \\right)}\\)\nIf we want a numerical approximation, we can use the evalf method:\nintegrate(expr, (x, 0, 5)).evalf() \\(\\displaystyle 1.37340076694502\\)\nBy the way, the documentation of the method has a section dedicated just to the approximation algorithms used for integrals. If you enjoyed the beginning of the article about the Riemann method, you may be interested in reading the documentation.\nSpeaking of approximation methods\u0026hellip;\nIntegrals with SciPy # We have written a lot about SymPy here on the site. The great power of SymPy is that it is a symbolic library, so we can enter and obtain results in a way very similar to what we are used to in everyday materials, such as books. You must have noticed this while reading the article: the notation used is very close to what we see in books and papers. The very fact that the result of the definite integral is presented as atan(5) instead of its approximate numerical value is quite characteristic of a symbolic library.\nFrom my point of view, it is very useful to start studying scientific Python with a symbolic library, hence the number of articles I write about it. It is much simpler to create the symbols and manipulate them as in a regular math course or similar, without the need to keep creating lists or arrays of numbers and functions.\nHowever, everything has its pros and cons. SymPy can be relatively slow when used to obtain numerical values from very long computations. And it is not focused on numerical approximations, so it has few tools in that direction. Hence the existence of NumPy and of libraries built on top of it, such as SciPy. We will use the latter now.\nThe easiest way to get SciPy is through the Anaconda package, which we have already seen here on the site.\nOh, we have already shown SciPy here before when we computed the speed of the athlete Usain Bolt with Python. I recommend reading it ;-)\nSciPy is divided into subpackages that cover different domains of knowledge. In the integrate subpackage, intended for problems involving integrals and differential equations, there is the quad method. quad comes from quadrature, a historical term that refers to integrals.\nWe can pass a function and the limits of integration and obtain a numerical approximation of the result:\nfrom scipy.integrate import quad def example(x): return 1 / (x**2 + 1) quad(example, 0, 5) \\(\\displaystyle \\left( 1.37340076694502, \\ 7.16706990454181 \\cdot 10^{-9}\\right)\\)\nNote that a tuple was returned. The first value is the desired approximation; the second is an estimate of the error of this approximation. The value obtained for the approximation is equal to the one displayed by SymPy earlier when we asked for a numerical approximation. But the internal algorithm of quad is different from the one used by SymPy, being the Clenshaw-Curtis one.\nNote that when it was written above that we can pass a function, function is actually a Python function and not a function in the mathematical sense, as when we used SymPy. In SymPy, we created a variable that represents a mathematical function, making everything more fluid. Here, with SciPy, we need to create a Python function that represents the idea of a mathematical function. Be careful with the meaning of the same word in different contexts so you don\u0026rsquo;t get confused.\nBut let\u0026rsquo;s recall the variable that represented our function back in SymPy:\nexpr \\(\\displaystyle \\frac{1}{x^{2} + 1}\\)\nBy default, SymPy objects do not communicate with SciPy objects. However, SymPy has the lambdify function, which lets us transform SymPy objects into objects that can be understood by SciPy and NumPy. We pass the symbol used in the object and the SymPy object itself:\nfrom sympy import lambdify f = lambdify(x, expr) f \u0026lt;function _lambdifygenerated(x)\u0026gt; type(f) function Note that the return is a function-type object. Let\u0026rsquo;s pass it to quad:\nquad_approx = quad(f, 0, 5) quad_approx \\(\\displaystyle \\left( 1.37340076694502, \\ 7.16706990454181 \\cdot 10^{-9}\\right)\\)\nWe got the same result. lambdify is incredibly useful in this symbolic -\u0026gt; numerical transition.\nI wrote above that there are several approximation methods in SciPy. At the beginning of the article, we saw the Riemann sum, showing the idea of ever narrower rectangles approaching the area under the curve. And I mentioned other approximation methods: trapezoid and Simpson. Let\u0026rsquo;s see the implementation of these methods with SciPy.\nStarting with the trapezoid rule. The idea is the same as the one seen in the Riemann animation but, instead of rectangles, trapezoids are used. So, we need points that will serve as the vertices of these trapezoids. Let\u0026rsquo;s start by creating 10 values for x and their corresponding function values. We will use NumPy\u0026rsquo;s linspace to generate 10 evenly spaced values between our limits of integration 0 and 5:\nimport numpy as np x = np.linspace(0, 5, 10) x, f(x) (array([0. , 0.55555556, 1.11111111, 1.66666667, 2.22222222, 2.77777778, 3.33333333, 3.88888889, 4.44444444, 5. ]), array([1. , 0.76415094, 0.44751381, 0.26470588, 0.16839917, 0.11473088, 0.08256881, 0.06202144, 0.0481856 , 0.03846154])) Now, we will see the estimate given by SciPy\u0026rsquo;s trapezoid method:\nfrom scipy.integrate import trapezoid trap_approx = trapezoid(f(x), x) trap_approx \\(\\displaystyle 1.37305961354302\\)\nLet\u0026rsquo;s see the difference between the value obtained by this method and the one obtained by the quad method:\ntrap_approx - quad_approx[0] \\(\\displaystyle -0.00034115340199925\\)\nWe see that, even with only 10 points, the error only shows up from the fourth decimal place on. And if we have more points?\npoints = (5, 10, 20, 50, 100, 200) print(\u0026#39;Approximation Difference vs. quad\u0026#39;) for count in points: x = np.linspace(0, 5, count) trap_approx = trapezoid(f(x), x) print(trap_approx, trap_approx - quad_approx[0]) Approximation Difference vs. quad 1.3922446845579102 0.018843917612893613 1.3730596135430173 -0.00034115340199925015 1.3733154390709228 -8.532787409376574e-05 1.3733879321834697 -1.2834761546898221e-05 1.3733976225738 -3.1443712165124538e-06 1.3733999887221857 -7.782228308439443e-07 As expected, the more points, the smaller the difference between the methods. Since the approximation given by quad uses a more elaborate form of computation, we can take it as a reference.\nThe simpson method follows the same logic, but using the Simpson rule mentioned earlier:\nfrom scipy.integrate import simpson print(\u0026#39;Approximation Difference vs. quad\u0026#39;) for count in points: x = np.linspace(0, 5, count) simpson_approx = simpson(f(x), x) print(simpson_approx, simpson_approx - quad_approx[0]) Approximation Difference vs. quad 1.3086914096492739 -0.06470935729574268 1.3697275044800366 -0.0036732624649800094 1.373392141184242 -8.625760774538804e-06 1.3734007318507901 -3.50942264137899e-08 1.373400764926956 -2.0180606075115293e-09 1.3734007668239518 -1.2106471380946005e-10 Likewise, the difference shrinks the more points are provided.\nConclusion # I hope the article has helped you review calculus and integrate your mathematical knowledge with your Python programming knowledge. And, moreover, that it has made you aware that almost every value presented by a program is the result of approximations. Forgetting this fact can lead to problems. It is no accident that I call attention to approximations fairly often in the articles.\nSee you next time.\n","date":"2022-04-15","externalUrl":null,"permalink":"/2022/04/calculus-python-sympy-integrals/","section":"Blog","summary":"We have already written here on the site about the concepts of limit and derivative and how we can do the related computations with Python. Continuing our journey through calculus, in this article we will see how to work with integrals using SymPy and SciPy, Python packages. Along the way, we will pick up some notions of numerical methods.","title":"Integrals - Calculus with Python and SymPy","type":"posts"},{"content":"","date":"2022-04-15","externalUrl":null,"permalink":"/tags/max/","section":"tags","summary":"","title":"max","type":"tags"},{"content":"","date":"2022-04-15","externalUrl":null,"permalink":"/tags/min/","section":"tags","summary":"","title":"min","type":"tags"},{"content":"You may find it strange to have a whole post just for max. That is because its most common use is simply finding the highest number in a set of numbers. For example:\nlist_1_to_5 = [2, 1, 4, 5, 3] max(list_1_to_5) 5 But there are other uses, which we will see throughout this article.\nMore than one iterable # The function can receive more than one iterable. For example, two lists:\nlist_1_to_5 = [2, 1, 4, 5, 3] list_1_to_3 = [2, 3, 1] max(list_1_to_3, list_1_to_5) [2, 3, 1] Can you make sense of the result? The function compares the two lists item by item. The first item is the same in both lists. The second item is different, so the list with the larger second item was returned.\nThe key parameter # The real power of the max function is the key parameter. This parameter can receive a function that changes the condition for what is considered the maximum.\nUsing the len function to return the longest list:\nmax(list_1_to_3, list_1_to_5, key=len) [2, 1, 4, 5, 3] Using the sum function to return the list with the largest sum of its items:\nmax(list_1_to_3, list_1_to_5, key=sum) [2, 1, 4, 5, 3] What if there are repeated items? Using the default max function, the largest value is returned, as expected:\nlist_with_repeats = [1, 1, 2, 3, 4, 5, 6, 6, 2, 2] max(list_with_repeats) 6 But we can use methods such as a list\u0026rsquo;s count, which returns the number of occurrences of each item in the list. The number that appears most often in the list will be returned:\nmax(list_with_repeats, key=list_with_repeats.count) 2 Combining with itertools and operator # But what if you actually want the largest value among the two lists we created? You can join them and ask for the maximum value of the union. We can use the chain function from itertools, which is part of a standard Python installation:\nfrom itertools import chain max(chain(list_1_to_3, list_1_to_5)) 5 We have already seen that when there is more than one iterable, the default is to return the one with the largest first item. If those are all equal, the one with the largest second item, and so on:\nlist_of_tuples = [(1, 2, 3), (3, 1, 1), (8, 3, 3), (3, 4, 2)] max(list_of_tuples) (8, 3, 3) But what if we want to start from the largest second item? Or third? We can use itemgetter from operator, which is part of a standard Python installation:\nfrom operator import itemgetter max(list_of_tuples, key=itemgetter(1)) (3, 4, 2) max(list_of_tuples, key=itemgetter(2)) (1, 2, 3) Remember: in Python, counting starts at zero :-)\nWorking with dictionaries # What happens if a dictionary is passed? By default, it returns the largest key of the dictionary. In the example, the keys are strings, so it returns the string with the largest first letter (position in the alphabet):\nelements = {\u0026#39;zinc\u0026#39;: 30, \u0026#39;gold\u0026#39;: 79, \u0026#39;calcium\u0026#39;: 20, \u0026#39;hydrogen\u0026#39;: 1, \u0026#39;lead\u0026#39;: 82, } max(elements) 'zinc' If we want the criterion to be the largest value instead of the largest key, we can use the dictionary\u0026rsquo;s get method:\nmax(elements, key=elements.get) 'lead' Notice that the key with the largest associated value was returned. It is more useful to return the (key, value) pair. We can use an anonymous lambda function for that.\nReturning the (key, value) pair with the largest key:\nmax(elements.items(), key=lambda k: k[0]) ('zinc', 30) Returning the (key, value) pair with the largest value:\nmax(elements.items(), key=lambda k: k[1]) ('lead', 82) Working with numbers in string form # To finish, we have already seen that with strings the largest positional value is returned:\nstrings = [\u0026#39;1\u0026#39;, \u0026#39;100\u0026#39;, \u0026#39;111\u0026#39;, \u0026#39;2\u0026#39;] max(strings) '2' The result was \u0026lsquo;2\u0026rsquo; because it is the largest value when comparing the first positions of each string. But if we want the largest integer, we just use the int function as the key:\nmax(strings, key=int) '111' Conclusion # Did you know about all these possible uses of the max function? If we missed an interesting use, let us know in the comments.\nThe key parameter is a classic example of why it is important to read the documentation when you use a function for the first time. We end up discovering more features than the one that shows up most obviously on first use.\nOh, and everything shown in this post applies in the same way to the min function ;-)\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the \u0026quot;drops\u0026quot; tag here on the site.\nUntil next time!\n","date":"2022-04-15","externalUrl":null,"permalink":"/2022/04/max-function-in-python/","section":"Blog","summary":"Did you know that Python’s max function has many options and ways of use that few people know about? In this post we will see different ways of using the max function in Python.","title":"The max Function in Python","type":"posts"},{"content":"","date":"2022-04-07","externalUrl":null,"permalink":"/tags/career/","section":"tags","summary":"","title":"career","type":"tags"},{"content":"","date":"2022-04-07","externalUrl":null,"permalink":"/tags/carreira/","section":"tags","summary":"","title":"carreira","type":"tags"},{"content":"","date":"2022-04-07","externalUrl":null,"permalink":"/tags/communication/","section":"tags","summary":"","title":"communication","type":"tags"},{"content":"","date":"2022-04-07","externalUrl":null,"permalink":"/tags/comunicacao/","section":"tags","summary":"","title":"comunicação","type":"tags"},{"content":"","date":"2022-04-07","externalUrl":null,"permalink":"/tags/git/","section":"tags","summary":"","title":"Git","type":"tags"},{"content":"I recently watched this excellent video by Fabio Akita for Alura (in Portuguese). The video is about best practices in programming. But the idea can be broadened to best practices in general, for any career. And, deep down, it is a video about communication, as I will show in this article.\nDon\u0026rsquo;t know anything about programming? No problem, the text doesn\u0026rsquo;t require it, it is about careers. But if you are a programmer or interested in the field, at the end I tell you which language you should learn. Come along, I will start by talking a bit about my own career.\nBest practices in my career\u0026hellip; as a chemist! # I trained as a chemist and I understand well what it means for best practices to evolve over time, one of the ideas presented in the video. Time goes by and people begin to realize, because of experiences that are not always pleasant, that some practices are not the best, for example, smoking around laboratories with vapors of flammable solvents!\nAnd more than that. In the past, access to equipment for chemical analysis was more restricted and expensive, and the analyst had to think carefully about which techniques were the bare minimum needed to characterize the object of study. To keep the text easy to understand for those who are not chemists, I will spare the technical terms, but know that, at times, \u0026ldquo;manual\u0026rdquo; laboratory techniques were chosen over instruments because of cost or access. Today, most laboratories that need it have access to basic equipment, freeing up the time of technicians and chemists for activities that generate more value, making them more productive.\nIt is no coincidence that a significant part of the training of a technician or graduate today is devoted to the study of instrumental analysis. Some forms of bench characterization that were best practices in the 80s and 90s have now been replaced by others, carried out with instruments.\nBest practices in the digital world # OK, but you, the reader, may not be a chemist. So what? Where are we going with this text?\nWell, if analyses are performed by instruments, the results are delivered as files, right? Of the most varied types and with the most varied contents, depending on which analysis is being done. And such files are shared by a whole team of analysts.\nHere arises the need for practices that made no sense in the past. For example: what is the file naming standard? Will each analysis have its own folder? Will files be separated by the date of the analysis? Will the \u0026ldquo;raw\u0026rdquo; data files stay in the same folder as the final analyses? And those final analyses, if they go through review, how will the version control system work? Will there be a Standard Operating Procedure (SOP) for this type of analysis? It is no coincidence that today there are several ISOs and standards that address such points, which are important for the reproducibility of experiments and for auditing. Best practices evolve over time\u0026hellip;\nSOP: Standard Operating Procedure. Don\u0026rsquo;t believe the topic is relevant? See the result of a simple search for \u0026ldquo;standard operating procedure\u0026rdquo; (in Portuguese) on Amazon. A multitude of books on the subject, for several fields.\nDid you recognize this need in your career too? Do you deal with recurring procedures? Do you deal with file after file all the time? Do these files need to be shared with a whole team that needs to understand the context of each one? Do you agree with me that it is a matter of communication? What is the impact if there is no effective communication in the team about shared resources and digital data?\nLook, even if you are not in programming, these are questions that arise from technology entering your routine, which creates the need for ways to deal with them. And of course the demand for best practices and effective communication already existed before, it was just amplified in the current technological context.\nEffective communication # The last point is what gives this section its name:\nCommunication. How did a video about best practices in programming give rise to this text? Simple, every field has the concept of \u0026ldquo;best practices\u0026rdquo;. It matters little whether you are an engineer, architect, geographer, physicist, anthropologist, musician, \u0026hellip; Surely, the term \u0026ldquo;best practices\u0026rdquo; is in the air of every profession. And they can change as the scenario changes, as in the example presented, through technological developments.\nAnd, considering what was described in earlier paragraphs, many professions today also deal with (an excess of) digital information. You may have no idea what Git or \u0026ldquo;variable name\u0026rdquo; is but, surely, you have seen some understanding problem in your team caused by using old versions of files or by a file having a name that was not informative. Or by thinking that a given instruction in a document meant A when, in fact, it is B. That\u0026rsquo;s it.\nI already showed that several fields have SOPs. The example I gave above about \u0026ldquo;variable names\u0026rdquo; is quite specific to programming. Within this field, there are also several attempts to define best practices for writing code. Variable and function names are only a small part, there are many aspects to writing code that communicates what is expected of the program to everyone involved in the development.\nThat is what much of the video is about: effective communication, best practices to make the team understand the context of a problem, best practices for the effective use of shared resources, best practices of language.\nProgramming languages and programs # \u0026ldquo;Language\u0026rdquo;\u0026hellip; Is everything presented as a best practice really one? Are there universal best practices or do they arise locally? Or a bit of both? Are the best practices described in standards enough?\nWatching the video with a more attentive eye, we see that some discussions arise because people treat as dogmas what are, in reality, flexible practices or tools that can (should) be used if, and only if, they make sense in the context. To end this text, we will, yes, talk a little about programming just to illustrate a personal experience. But what I am going to say is brief and general enough for everyone to understand.\nI already told how I observed the evolution in chemistry, my field of training, with regard to equipment. But, of course, this evolution was accompanied by the emergence of the most diverse software to analyze the data generated by the instruments. From the infamous spreadsheets, through specific programs for statistical data processing, to the use of programming languages to handle the absurd amount of data generated and to make simulations and predictions. I enjoyed this evolution so much, and learned so much, that today I am moving into data science, but that is talk for another article. The point is: I saw many \u0026ldquo;dogmas\u0026rdquo; emerge that, in reality, were momentary and circumstantial best practices and, today, are replaced by other best practices depending on the context.\nJust one example to illustrate. Talking to an academic colleague once, I said I was using Python for a certain study. The colleague asked \u0026ldquo;What is Python?\u0026rdquo;. And I explained that it is a programming language that has several interesting packages for statistics and for presenting the results of the study I was doing. The immediate answer was \u0026ldquo;This is wrong. FORTRAN is for that, it is faster, it is the standard, and it has everything you need. I don\u0026rsquo;t understand these fads of using different languages\u0026rdquo;. Wow\u0026hellip; did you see the dogmatic attitude?\nFirst of all, I have a special fondness for FORTRAN, it was the first programming language I used regularly. I am very happy that it still has a community and keeps evolving. And I started using Python because\u0026hellip; the main scientific libraries of Python use FORTRAN under the hood! Take a look at the SciPy FAQ page, for example. But if under the hood it is FORTRAN, why not use FORTRAN directly? Simple, the problem I wanted to solve required other things besides the incredible numerical capability of FORTRAN. I wanted to integrate with other tools and integration was simpler with Python. I felt more productive with Python. The code was easier to understand and maintain with Python. The audience that would come into contact with the result was more familiar with Python. Therefore, in an analysis of the whole life cycle of what was being produced, the best alternative for that situation was\u0026hellip; Python.\nAnd what I said here about Python vs FORTRAN can be replicated in many contexts involving other programming languages or software. I see engineers having analogous discussions involving Matlab vs Python vs R. Visual folks in discussions about Photoshop vs GIMP, Premiere vs DaVinci Resolve. Why? Why must you choose only one of the options? Is any of them really a silver bullet? Is the option itself more important than the problem to be solved?\nWhat is the best programming language? # We are in a moment of digital abundance. There are many choices of digital tools and programming languages. Every field today produces more and more data that needs to be analyzed for decision making. The market is flooded with buzzwords such as artificial intelligence, data science, machine learning, personal branding, digital marketing. Flooded with promises that by mastering these fields your business will take off or you will easily enter the job market. Flooded with courses on all sorts of programs for all sorts of purposes. Flooded with programming courses and the like, from Excel VBA and SQL to Javascript, R, Python.\nWhat should you learn? What is the best practice of the moment? Which tool? Which language should you learn? Answer: English. Or Portuguese, or Spanish. Or smoke signals, for that matter. That is the best practice, the tool: your ability to communicate, the concern that others understand your message, your work, and that you understand theirs. The language that will allow you to communicate effectively with your teammates and clients to understand the problem to be solved. That will allow you to define the best practices for those involved in the context, at the time the problem presents itself. That will allow you to define which language, now yes, programming language, or program or any other tool is the most suitable for the situation.\nIf you guide your career by tools, the tool will be you. Tools and best practices change. What does not change is the need to be able to recognize a problem and present solutions based on the available resources. And, in this process, communication is fundamental.\nSee you next time.\n","date":"2022-04-07","externalUrl":null,"permalink":"/2022/04/what-are-best-practices/","section":"Blog","summary":"I recently watched an excellent video about best practices in programming. But the idea can be broadened to best practices in general, for any career. And, deep down, it is a video about communication, as I will show in this article.","title":"So, What Are Best Practices?","type":"posts"},{"content":" Engineers and scientists are always trying to get the most performance out of their equipment and projects at the lowest cost. This search gives rise to so-called optimization problems. In this article, we will see how to use Python, with the help of the SymPy package, to solve them.\nUnderstanding an optimization problem # Let\u0026rsquo;s start by understanding the essence of an optimization problem from a mathematical point of view. We can think of optimization as the search for an input to a function \\(f(x)\\) such that the result is the best output value of \\(f(x)\\). And what would this \u0026ldquo;best value\u0026rdquo; be? Usually, it means the maximum value (if the function represents something positive, such as profit) or the minimum value (if the function represents something undesirable, such as cost).\nIn the article on derivatives with Python, we saw that the derivative of a function is related to the slope of a line tangent to the function at a given point. Thus, when \\(f^{\\prime}(x) = 0\\), that slope is zero, indicating that we are at a maximum or a minimum. Points that satisfy the previous equality are called critical points of a function.\nThe second derivative, \\(f^{\\prime\\prime}(x)\\), is related to the curvature of \\(f(x)\\), and we can use it to check whether a given critical point is a maximum or a minimum. If \\(f^{\\prime\\prime}(x) \u0026lt; 0\\), the function has a maximum; if \\(f^{\\prime\\prime}(x) \u0026gt; 0\\), the function has a minimum; and if \\(f^{\\prime\\prime}(x) = 0\\), the function is at an inflection point, changing from a region of maximum to one of minimum, or vice versa.\nLet\u0026rsquo;s go through an example. Consider the following function:\n$$ f(x) = x^3 - 2x^2 + x $$Let\u0026rsquo;s create it with SymPy:\n# setup for better outputs in the article, can be ignored from sympy import init_printing init_printing(scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) # importing what we need for the start of this article from sympy import Symbol, diff, solve, plot # creating the variable symbol and the function x = Symbol(\u0026#39;x\u0026#39;) f = x**3 - 2*x**2 +x f \\(\\displaystyle x^{3} - 2 x^{2} + x\\)\nWith SymPy\u0026rsquo;s diff function, we can easily obtain the first and second derivatives:\nfirst_derivative = diff(f, x) first_derivative \\(\\displaystyle 3 x^{2} - 4 x + 1\\)\nsecond_derivative = diff(f, x, 2) second_derivative \\(\\displaystyle 2 \\left(3 x - 2\\right)\\)\nWith the solve function, we can obtain the critical points by solving the first derivative for x. That is, by solving \\(f^{\\prime}(x)=0\\):\ncritical_points = solve(first_derivative, x) critical_points \\(\\displaystyle \\left[ \\frac{1}{3}, \\ 1\\right]\\)\nNote that the result comes out as a list. We can substitute each result into the expression of the second derivative:\nsecond_derivative.subs({x: critical_points[0]}), second_derivative.subs({x: critical_points[1]}) \\(\\displaystyle \\left( -2, \\ 2\\right)\\)\nAnd check whether there is any inflection point:\ninflection_point = solve(second_derivative, x) inflection_point \\(\\displaystyle \\left[ \\frac{2}{3}\\right]\\)\nWe see that \\(f^{\\prime\\prime}(\\frac{1}{3}) \u0026lt; 0\\), indicating a maximum, and that \\(f^{\\prime\\prime}(1) \u0026gt; 0\\), a minimum. At \\(x = \\frac{2}{3}\\) the concavity of the function changes.\nLet\u0026rsquo;s look at the graph of the function:\nplot(f, (x, -0.2, 1.2)); Note that the graph was deliberately built over an interval (domain) in which the maximum and the minimum are evident. However, observe that the function is increasing for \\(x \u0026gt; 1\\) and decreasing for \\(x \u0026lt; 0\\). Hence the importance of defining a domain of validity for the problem right at the start. Assuming a domain such that \\(x \\in [0, 1]\\), we can state that \\((\\frac{1}{3}, f(\\frac{1}{3}))\\) is a maximum.\nApplication example: minimizing costs # Let\u0026rsquo;s take a very common example in chemical processes: minimizing the cost of building a tank. By the way, I am a chemist by training, and I have already written here a bit about how technology influenced my career, just as it certainly influences yours. For more details on my projects, see my portfolio.\nObtaining our function # In our example, we will consider a cylindrical tank of fixed volume (V), and we want to determine the length (L) and the diameter (D) that minimize the cost of the tank. The cost per unit area of the side of the tank is \\(c_s\\), and the cost per unit area of the top and bottom of the tank is \\(c_t\\). With a bit of basic geometry, shown in the following figure, we can obtain our function \\(f\\), which represents the cost:\nLet\u0026rsquo;s create the symbols needed to represent our function. Note that all of them were given the restriction of being positive values, since it makes no sense to talk about negative dimensions or costs. It is important to remember to evaluate restrictions in optimization problems.\nfrom sympy import pi diameter = Symbol(\u0026#39;D\u0026#39;, positive=True) length = Symbol(\u0026#39;L\u0026#39;, positive=True) volume = Symbol(\u0026#39;V\u0026#39;, positive=True) side_cost = Symbol(\u0026#39;c_s\u0026#39;, positive=True) top_cost = Symbol(\u0026#39;c_t\u0026#39;, positive=True) f = side_cost * pi * diameter * length + top_cost * (pi / 2) * diameter**2 f \\(\\displaystyle \\frac{\\pi D^{2} c_{t}}{2} + \\pi D L c_{s}\\)\nLet\u0026rsquo;s assume there is no limit on the values and look for an expression in terms of V and D. Thus, eliminating L from the expression:\ncylinder_volume = (pi / 4) * diameter**2 * length cylinder_volume \\(\\displaystyle \\frac{\\pi D^{2} L}{4}\\)\nlength_from_volume = solve(cylinder_volume - volume, length)[0] length_from_volume \\(\\displaystyle \\frac{4 V}{\\pi D^{2}}\\)\nf = f.subs({length: length_from_volume}) f \\(\\displaystyle \\frac{\\pi D^{2} c_{t}}{2} + \\frac{4 V c_{s}}{D}\\)\nTherefore, the expression above is the one we want to optimize, obtaining a minimum value. Let\u0026rsquo;s obtain the expression of its first derivative with respect to the diameter:\nf_min = diff(f, diameter) f_min \\(\\displaystyle \\pi D c_{t} - \\frac{4 V c_{s}}{D^{2}}\\)\nNow, we can obtain an expression for the diameter that makes the derivative zero:\noptimal_diameter = solve(f_min, diameter)[0] optimal_diameter \\(\\displaystyle \\frac{2^{\\frac{2}{3}} \\sqrt[3]{V} \\sqrt[3]{c_{s}}}{\\sqrt[3]{\\pi} \\sqrt[3]{c_{t}}}\\)\nAnd, with the diameter, obtain the optimal length:\noptimal_length = length_from_volume.subs({diameter: optimal_diameter}).simplify() optimal_length \\(\\displaystyle \\frac{2^{\\frac{2}{3}} \\sqrt[3]{V} c_{t}^{\\frac{2}{3}}}{\\sqrt[3]{\\pi} c_{s}^{\\frac{2}{3}}}\\)\nLet\u0026rsquo;s see which length / diameter ratio minimizes the cost:\noptimal_length / optimal_diameter \\(\\displaystyle \\frac{c_{t}}{c_{s}}\\)\nNote how interesting this result is. If the costs are the same, the cylinder that minimizes the cost is the one whose length equals the diameter of its base, since the ratio above would be 1. We will write more about this below.\nLet\u0026rsquo;s plug in example values to better visualize our result. We will consider the costs equal to 1 unit and the volume equal to 10 units. The diameter that minimizes the function is:\nsolve(f_min.subs({side_cost: 1, top_cost: 1, volume: 10}), diameter)[0] \\(\\displaystyle \\frac{2 \\sqrt[3]{5}}{\\sqrt[3]{\\pi}}\\)\nWhich, numerically, corresponds to:\n# the _ lets us retrieve the value of the previous cell _.n() \\(\\displaystyle 2.33508864988147\\)\nLet\u0026rsquo;s look at the graph of our cost function:\np = plot(f.subs({side_cost: 1, top_cost: 1, volume: 10}), (diameter, 0, 5), ylim=(0, 100), legend=True, show=False) p[0].label = \u0026#39;f\u0026#39; p.show() Note how there really is a minimum around a diameter of 2.33. Therefore, this is the diameter (and also the length) for the given boundary conditions.\nWhat if the costs are different? # What happens if the costs are different? Let\u0026rsquo;s consider the cost of the side of the tank to be twice the cost of the top:\nsolve(f_min.subs({side_cost: 2, top_cost: 1, volume: 10}), diameter)[0] \\(\\displaystyle \\frac{2 \\sqrt[3]{10}}{\\sqrt[3]{\\pi}}\\)\n_.n() \\(\\displaystyle 2.94202734335627\\)\nNote that the diameter of the base increased compared to the previous case. Consequently, since the length / diameter ratio depends only on the costs, the length is half the diameter obtained. Can you now understand the \u0026ldquo;flattened\u0026rdquo; shape of the cylinders that store liquids, which we usually see in industrial plants?\nIn these cylinders, the internal pressure of the liquid is given by the famous equation \\(P = \\rho g h\\), that is, it depends on the density of the liquid, on gravity, and on the height of the liquid column. Therefore, a tall tank would require thicker walls to withstand the pressure of the liquid. And, of course, this greatly affects construction costs, especially of the side walls. Simple, right? Of course, there is much more involved, as can be seen here.\nAnd what about all those different shapes of food cans? # We saw in the calculations above that, if the costs of the side and of the top are equal, the cylindrical shape that minimizes the total cost is the one whose base diameter equals the length of the cylinder. But then why is that shape so rare among the canned goods we see on the market? In fact, first of all, why a cylinder?\nIn our calculations, we implicitly considered that the cost comes only from the area. So why not use spherical containers? After all, one of the first things taught in geometry classes is that the sphere has the smallest surface area of all closed shapes around a given volume. OK, cool\u0026hellip; Now try stacking several spherical containers on a supermarket shelf. Good luck :-)\nBesides the fact that spheres roll very easily, their packing is inefficient. At most, we can fill 74% of the available space by stacking spheres. Look for your nearest trusted chemist if you want a mathematical proof of that. Thus, it is clear that other considerations are involved. It is not just the construction cost; there is also the cost of transport, storage, and disposal. In this sense, more cylindrical shapes turn out to be more interesting. Besides the ergonomic factor, your customer needs to be able to hold what they are going to consume. Can you imagine having to hold a soda can with both hands because its diameter is large?\nIn the can manufacturing process, the greatest waste is at the top and bottom, because of the need for cutting and welding. Of course, there is recycling, but that makes the cost different, and in that case the cost of the side is lower, which makes taller cans make sense.\nConclusion # Besides calculus and programming, we also had a crash course on packaging ;-)\nWe saw the care that must be taken in optimization problems, from the correct identification of the domain, through the boundary conditions, to the critical analysis of the results obtained. And the whole process was quite agile thanks to SymPy.\nSee you next time.\n","date":"2022-04-05","externalUrl":null,"permalink":"/2022/04/optimization-problems-python/","section":"Blog","summary":"Engineers and scientists are always trying to get the most performance out of their equipment and projects at the lowest cost. This search gives rise to so-called optimization problems. In this article, we will see how to use Python, with the help of the SymPy package, to solve them.","title":"Optimization Problems with Python","type":"posts"},{"content":"","date":"2022-04-01","externalUrl":null,"permalink":"/categories/linux/","section":"Categories","summary":"","title":"Linux","type":"categories"},{"content":"","date":"2022-04-01","externalUrl":null,"permalink":"/tags/linux-mint/","section":"tags","summary":"","title":"Linux Mint","type":"tags"},{"content":"Linux Mint is a distribution based on Ubuntu, and its main desktop environment is Cinnamon. It has a look that is more welcoming for people migrating from Windows, and it meets the needs of all kinds of users.\nIn this video (in Portuguese), we will see how to customize Linux Mint.\nVideo contents # Zoom tool Caps Lock and Num Lock activation indicator More visible mouse cursors Installing and using themes Changing the panel position and customizing the panel Customizing the date and time display Applets Changing the wallpaper Extensions: transparent panel ScreenKey: showing the pressed keys Extensions: gTile, arranging windows efficiently Shortcuts: switching workspaces Creating more workspaces Shortcuts: moving windows between workspaces Shortcuts: arranging windows on the screen Shortcuts: creating your own shortcuts Shortcuts: a few more useful ones Changing the behavior of Caps Lock and ESC Enabling RedShift Changing the LibreOffice panels to look like Microsoft Office Enabling/disabling system sounds System monitor Renaming multiple files in the Nemo file manager Additional links # Linux Mint website\nLink to the cursor installed in the video\nLink to the ScreenKey website\nInstalling Linux Mint # We already made a video showing in detail how to install Linux Mint (in Portuguese). In it, we show how to do an encrypted installation on a computer with more than one storage drive:\nLinux Mint and Programming # Here at Ciência Programada we write a lot about programming. We already made a video (in Portuguese) showing how to install the Anaconda distribution on Linux Mint to program with Python:\nLink to the article about Anaconda.\nWe also have a video and an article showing how to create virtual environments for Python projects on Linux Mint.\n","date":"2022-04-01","externalUrl":null,"permalink":"/2022/04/linux-mint-after-install-customization/","section":"Blog","summary":"Linux Mint is a friendly, easy-to-use Linux distribution. In this video (in Portuguese), we make a series of customizations that make it even friendlier and more productive.","title":"Linux Mint: What to Do After Installing? Complete Customization Guide","type":"posts"},{"content":"","date":"2022-03-31","externalUrl":null,"permalink":"/tags/string/","section":"tags","summary":"","title":"string","type":"tags"},{"content":"We spend a considerable amount of time dealing with strings, so it is important to know how to present them the way we want. That is what we will see in this article, with plenty of examples.\nPlaceholder # This is the oldest way of formatting strings in the language. Nowadays, it is recommended to use the other forms presented later in the article. However, since there is still a lot of legacy code out there, it is important to know how to use it.\nAs the name suggests, it uses placeholders, such as %s, inside the string. For example:\nprint(\u0026#39;A text will be placed %s\u0026#39; % \u0026#39;here\u0026#39;) A text will be placed here When more than one insertion is needed, a tuple is passed with each insertion in the order it must appear in the string:\nprint(\u0026#39;A text will be placed %s, and %s too\u0026#39; % (\u0026#39;here\u0026#39;, \u0026#39;more here\u0026#39;)) A text will be placed here, and more here too Sometimes we need to pass a string that has a special meaning for the language interpreter, but we want the interpreter to treat it as a normal string. We call these cases raw strings, literally \u0026ldquo;uncooked\u0026rdquo; strings. For example, \\t in Python represents the space of the TAB key. If we want \\t to be displayed literally instead of the space, we indicate it with %r, where the r stands for raw string:\nprint(\u0026#39;Here is the TAB space: \\t. Now the raw representation: %r.\u0026#39; % \u0026#39;\\t\u0026#39;) Here is the TAB space: . Now the raw representation: '\\t'. Dealing with numbers # If you want a faithful representation of the number passed, use %s so that it is converted to a string. But manipulations can be done, such as truncating to an integer and showing the sign:\nnum = 13.744 print(\u0026#39;Numbers like %s are converted to string\u0026#39; % num) print(\u0026#39;They can also be represented as integers: %d; note there is no rounding\u0026#39; % num) print(\u0026#39;Showing the sign in integers: %+d\u0026#39; % num) print(\u0026#39;Showing the sign in floats: %+f\u0026#39; % num) Numbers like 13.744 are converted to string They can also be represented as integers: 13; note there is no rounding Showing the sign in integers: +13 Showing the sign in floats: +13.744000 Note above that the float representation has more decimal places than the number passed. We can change this representation with x.yf, where x is the minimum number of characters and y, the decimal places:\nprint(\u0026#39;Minimum of 5 characters with 2 decimal places: %5.2f\u0026#39; % num) print(\u0026#39;Minimum of 1 characters with 0 decimal places: %1.0f (note the rounding)\u0026#39; % num) print(\u0026#39;Minimum of 1 characters with 5 decimal places: %1.5f\u0026#39; % num) print(\u0026#39;Minimum of 10 characters with 2 decimal places: %10.2f\u0026#39; % num) print(\u0026#39;Minimum of 10 characters with 2 decimal places: %+10.2f (with sign)\u0026#39; % num) Minimum of 5 characters with 2 decimal places: 13.74 Minimum of 1 characters with 0 decimal places: 14 (note the rounding) Minimum of 1 characters with 5 decimal places: 13.74400 Minimum of 10 characters with 2 decimal places: 13.74 Minimum of 10 characters with 2 decimal places: +13.74 (with sign) The format method # This form was introduced in Python 3.0 and is more flexible and more readable than the previous one.\nWith a single insertion, it looks like this:\nprint(\u0026#39;A text will go in {}.\u0026#39;.format(\u0026#39;here\u0026#39;)) A text will go in here. A great improvement of format is being able to choose the order of insertion and to work with variables:\nprint(\u0026#39;Controlling {2} {1}{0}\u0026#39;.format(\u0026#39;!\u0026#39;, \u0026#39;order\u0026#39;, \u0026#39;the\u0026#39;)) print(\u0026#39;First: {a}; Second: {b}; Third: {c}\u0026#39;.format(a=1, b=\u0026#39;2\u0026#39;, c=12.3)) print(\u0026#39;The variable {p} can be reused: {p}.\u0026#39;.format(p=\u0026#39;(hi!)\u0026#39;)) Controlling the order! First: 1; Second: 2; Third: 12.3 The variable (hi!) can be reused: (hi!). We can even use dictionary keys through unpacking, as in the following example:\nperson = {\u0026#39;name\u0026#39;: \u0026#39;Chico\u0026#39;, \u0026#39;age\u0026#39;: 33} print(\u0026#39;Hello, {name}. You are {age} years old.\u0026#39;.format(**person)) Hello, Chico. You are 33 years old. Alignment # With format there was a great advance in the possibilities of displaying text output. Below, we see how to indicate the space to be occupied by each insertion:\nprint(\u0026#39;{0:16} | {1:5}\u0026#39;.format(\u0026#39;Name\u0026#39;, \u0026#39;Age\u0026#39;)) print(\u0026#39;{0:16} | {1:5}\u0026#39;.format(\u0026#39;Chico\u0026#39;, 33)) print(\u0026#39;{0:16} | {1:5}\u0026#39;.format(\u0026#39;Brás Cubas\u0026#39;, 64)) print(\u0026#39;{0:16} | {1:5}\u0026#39;.format(\u0026#39;Machado de Assis\u0026#39;, \u0026#39;69\u0026#39;)) Name | Age Chico | 33 Brás Cubas | 64 Machado de Assis | 69 See how the numeric values were aligned to the right, while the strings were aligned to the left. We can control the alignment as follows:\nprint(\u0026#39;{0:\u0026lt;8} | {1:^8} | {2:\u0026gt;8}\u0026#39;.format(\u0026#39;Left\u0026#39;, \u0026#39;Center\u0026#39;, \u0026#39;Right\u0026#39;)) print(\u0026#39;{0:\u0026lt;8} | {1:^8} | {2:\u0026gt;8}\u0026#39;.format(11, 22, 33)) Left | Center | Right 11 | 22 | 33 It is also possible to fill the space with some character:\nprint(\u0026#39;{0:=\u0026lt;8} | {1:-^8} | {2:.\u0026gt;8}\u0026#39;.format(\u0026#39;Left\u0026#39;, \u0026#39;Center\u0026#39;, \u0026#39;Right\u0026#39;)) print(\u0026#39;{0:=\u0026lt;8} | {1:-^8} | {2:.\u0026gt;8}\u0026#39;.format(11, 22, 33)) Left==== | -Center- | ...Right 11====== | ---22--- | ......33 The alignment and size information can also be passed as parameters:\nprint(\u0026#39;\u0026#34;{:{align}{width}}\u0026#34;\u0026#39;.format(\u0026#39;text\u0026#39;, align=\u0026#39;^\u0026#39;, width=\u0026#39;10\u0026#39;)) \u0026quot; text \u0026quot; Dealing with numbers # Very similar to what we saw with placeholders:\nprint(\u0026#39;10 characters and 2 decimal places:{:10.2f}\u0026#39;.format(13.579)) 10 characters and 2 decimal places: 13.58 The novelty is the possibility of controlling the distance from the sign to the first digit:\nprint(\u0026#39;{:=5d}\u0026#39;.format(-42)) print(\u0026#39;{:=+5d}\u0026#39;.format(42)) # forcing the + to appear on a positive number - 42 + 42 And here we can also use parameters:\nprint(\u0026#39;10 characters and 2 decimal places:{:{sign}{width}.{prec}f}\u0026#39;.format(13.579, width=10, prec=2, sign=\u0026#39;+\u0026#39;)) 10 characters and 2 decimal places: +13.58 Dealing with dates # The format method makes it easy to format datetime objects:\nfrom datetime import datetime print(\u0026#39;{:%Y-%m-%d %H:%M}\u0026#39;.format(datetime(1969, 7, 20, 22, 56))) 1969-07-20 22:56 To learn more, see the format documentation.\nf-strings # Available since Python 3.6, f-strings advanced even further in flexibility and readability. Basic use is very simple: just put f (or F) before the string and the variable between braces:\nname = \u0026#39;Chico\u0026#39; print(f\u0026#39;Name: {name}.\u0026#39;) Name: Chico. F-strings are evaluated at runtime, so expressions can be passed directly:\nprint(f\u0026#39;Name: {name.upper()}\u0026#39;) Name: CHICO In the case above, we called the string method upper to present the name in uppercase. We can also pass mathematical operations:\nprint(f\u0026#39;{2 * 3}\u0026#39;) 6 This is so simple and handy that I will even throw in a multiplication table in 3 lines of code:\nprint(\u0026#39;Multiplication table of 2:\u0026#39;) for i in range(11): print(f\u0026#39;2 x {i:2} = {2 * i:2}\u0026#39;) Multiplication table of 2: 2 x 0 = 0 2 x 1 = 2 2 x 2 = 4 2 x 3 = 6 2 x 4 = 8 2 x 5 = 10 2 x 6 = 12 2 x 7 = 14 2 x 8 = 16 2 x 9 = 18 2 x 10 = 20 :-) See in the code above that it is also possible to pass instructions for string length and alignment. Here, I reserved two spaces for the numbers so that they would be aligned to the right.\nDealing with numbers # From personal experience, dealing with numbers in f-strings was a bit difficult at first. That is because what used to be the count of decimal places became the count of the total number of digits. Let us look at the following example, where I present the format method and the f-string:\nnum = 23.45678 print(\u0026#39;10 characters and 4 decimal places:{:10.4f} (format)\u0026#39;.format(num)) print(f\u0026#39;10 characters and 4 decimal places:{num:{10}.{6}} (f-string)\u0026#39;) 10 characters and 4 decimal places: 23.4568 (format) 10 characters and 4 decimal places: 23.4568 (f-string) Note in the code above that the 6 indicates the total number of digits: 2 before the decimal point and 4 decimal places. This is a bit confusing, but we will see a more intuitive way ahead.\nWhen we saw the previous forms of formatting, trailing zeros were added so that the number had the requested number of decimal places. This does not happen here:\nnum = 23.45 print(\u0026#39;10 characters and 4 decimal places:{:10.4f} (format)\u0026#39;.format(num)) print(f\u0026#39;10 characters and 4 decimal places:{num:{10}.{6}} (f-string)\u0026#39;) 10 characters and 4 decimal places: 23.4500 (format) 10 characters and 4 decimal places: 23.45 (f-string) These \u0026ldquo;oddities\u0026rdquo; can be easily worked around. Just use the formatting syntax we saw in the format method. It is recognized in f-strings! Like this:\nnum = 23.45 print(\u0026#39;10 characters and 4 decimal places:{:10.4f} (format)\u0026#39;.format(num)) print(f\u0026#39;10 characters and 4 decimal places:{num:10.4f} (f-string)\u0026#39;) 10 characters and 4 decimal places: 23.4500 (format) 10 characters and 4 decimal places: 23.4500 (f-string) Alignment # What we saw about alignment in the format section still holds here. And we can also use runtime evaluation. See the example:\npeople = { \u0026#39;chico\u0026#39;: {\u0026#39;name\u0026#39;: \u0026#39;Chico\u0026#39;, \u0026#39;age\u0026#39;: 33, \u0026#39;profession\u0026#39;: \u0026#39;Teacher\u0026#39;}, \u0026#39;bras\u0026#39;: {\u0026#39;name\u0026#39;: \u0026#39;Brás Cubas\u0026#39;, \u0026#39;age\u0026#39;: 64, \u0026#39;profession\u0026#39;: \u0026#39;Dead author\u0026#39;}, \u0026#39;machado\u0026#39;: {\u0026#39;name\u0026#39;: \u0026#39;Machado de Assis\u0026#39;, \u0026#39;age\u0026#39;: 69, \u0026#39;profession\u0026#39;: \u0026#39;Writer\u0026#39;}, } print(f\u0026#34;{\u0026#39;Name\u0026#39;:^16} | {\u0026#39;Age\u0026#39;:^6} | {\u0026#39;Profession\u0026#39;:^15}\u0026#34;) print(f\u0026#34;{\u0026#39;-\u0026#39;*16} | {\u0026#39;-\u0026#39;*6} | {\u0026#39;-\u0026#39;*15}\u0026#34;) for person in people: d = people[person] print(f\u0026#34;{d[\u0026#39;name\u0026#39;]:\u0026lt;16} | {d[\u0026#39;age\u0026#39;]:^6} | {d[\u0026#39;profession\u0026#39;]:\u0026gt;15}\u0026#34;) | Name | Age | Profession | | ---------------- | ----- | --------------- | | Chico | 33 | Teacher | | Brás Cubas | 64 | Dead author | | Machado de Assis | 69 | Writer | Dealing with dates # Just as when using format, it is also easy to format dates with f-strings:\nnow = datetime.now() print(f\u0026#39;{now:%Y-%m-%d %H:%M}\u0026#39;) 2022-03-24 15:47 To learn more, see the f-strings documentation.\nConclusion # We spend a considerable amount of time dealing with strings, so it is important to know how to present them the way we want.\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the tag \u0026ldquo;drops\u0026rdquo; here on the site.\nSee you next time!\n","date":"2022-03-31","externalUrl":null,"permalink":"/2022/03/string-formatting-python/","section":"Blog","summary":"We spend a considerable amount of time dealing with strings, so it is important to know how to present them the way we want. That is what we will see in this article, with plenty of examples.","title":"String formatting in Python","type":"posts"},{"content":" The start of a calculus course tends to feel abstract because of the concept of a limit, which we saw in a recent article. But it soon becomes more tangible with the introduction of the derivative, which has many obvious practical applications.\nIn this article, we will see how to work with derivatives using SymPy, a Python package.\nThe concept of the derivative and its interpretation # Let\u0026rsquo;s recall a bit of calculus to make the article easier to follow. Look at the animation below. It shows the curve of an arbitrary function, two marked points, A and B, and the triangle formed to obtain the change in the y axis quantity and the x axis quantity, with the help of a third point C. In the animation, the points get closer and closer. Notice that, in the limit where the points become closer, the hypotenuse of the triangle, initially a secant line of the curve, becomes a tangent line at the point of virtual meeting A. The blue line is the tangent at point A, shown precisely to display how the secant gets closer and closer to a tangent.\nAnimation showing a secant approaching a tangent Notice that the word limit appears in the text, and we have already written about the concept of limit. Basically, we see in the animation that the secant line approaches a tangent in the limit where points A and B get closer. We can formalize this better by writing that the slope of the tangent line to \\( (a, f(a)) \\) must be\n$$ \\lim\\limits_{h \\to 0} \\frac{f(a + h) - f(a)}{h} $$We say that a function \\(f\\) is differentiable at \\(a\\) if the limit above exists. In this case, the limit is denoted by \\(f^{\\prime}(a)\\) and called the derivative of \\(f\\) at \\(a\\). We will return to this idea of a tangent line and its consequences later in the article.\nBut so what? Well, this idea of a tangent line to a curve leads to many applications. It is the origin of the idea of instantaneous velocity in physics, for example. After all, we can picture instantaneity as two measurements in a very short time interval, virtually in the limit of this interval tending to zero. Just like the points in our animation. In fact, we have already written about how to use Python to determine Usain Bolt\u0026rsquo;s speed, the Olympic and world champion athlete, and in that article we used exactly this idea and a bit of calculus :-)\nIn this article we will focus more on the operational side, showing how to handle functions and their derivatives with SymPy. But it is always important to keep in mind the interpretation and applications of what we are doing.\nThe diff function # To begin, let\u0026rsquo;s consider a very simple function:\n$$ f(x) = x^3 $$From the simple differentiation rules studied in introductory calculus courses, we know that:\n$$ f'(x) = 3x^2 \\qquad f''(x) = 6x \\qquad f'''(x) = 6 $$But how do we obtain these results with SymPy? We need to use the package\u0026rsquo;s diff function. Let\u0026rsquo;s see, starting by importing what we need:\n# setup for better outputs in the article, can be ignored from sympy import init_printing init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) from sympy import diff, Symbol Let\u0026rsquo;s create the symbol needed to represent x and create our variable that will represent the function. Then we request the first derivative, passing the function and the variable with respect to which the function will be differentiated:\nx = Symbol(\u0026#39;x\u0026#39;) f = x**3 diff(f, x) \\( 3 x^{2} \\)\nSimple, right? And for higher-order derivatives, just pass the order as the third parameter. For example, for the second derivative:\ndiff(f, x, 2) \\(6 x\\)\nAnd for the third derivative:\ndiff(f, x, 3) \\(6\\)\nFor more complicated functions, in calculus we learn several differentiation rules, such as the product, quotient and chain rules, among others. SymPy handles such cases smoothly, directly giving the final result. See the examples below, compare the results with those obtained by hand applying differentiation rules, taking the chance to study calculus ;-)\nfrom sympy import sin, cos, exp # product rule diff(x**2 * sin(x), x) \\( 2 x \\sin{\\left(x \\right)} + x^{2} \\cos{\\left(x \\right)} \\)\n# chain rule diff(sin(x**2)) \\( 2 x \\cos{\\left(x^{2} \\right)} \\)\n# quotient rule diff(x**2 / sin(x), x) \\( - \\dfrac{x^2 \\cos(x)}{\\sin^2(x)} + \\dfrac{2x}{\\sin(x)} \\)\nNotice that the result above can be written as a single fraction, making the quotient rule more explicit. For that, we use the factor method already covered in another article of the SymPy series:\ndiff(x**2 / sin(x), x).factor() \\( - \\dfrac{ x (x \\cos(x) - 2 \\sin(x))}{\\sin^2(x)} \\)\nDifferential equations # Differential equations have many applications, making up a world of their own within calculus. They will certainly appear often here on the site. We have already seen in another article how to use the solve function to solve the common equations studied in high school. Now, let\u0026rsquo;s look at the dsolve function for differential equations.\nWe have already seen in this and other articles that we can define symbols in SymPy with the Symbol class or with the symbols function, to represent variables and constants in our expressions and equations. To make it explicit that something is a function, we can use the Function class. Let\u0026rsquo;s import what we need and show how to create a representation of a function f:\nfrom sympy import Function, dsolve, Eq f = Function(\u0026#39;f\u0026#39;) f(x) \\( f(x) \\)\nNow that we have our function, let\u0026rsquo;s create the differential equation\n$$ f(x) - f'(x) = 0 $$For that, we will use the Eq class and the diff function:\ndiff_eq = Eq(f(x) - diff(f(x), x), 0) diff_eq \\( f(x) - \\frac{d}{dx} f(x) = 0 \\)\nNotice above that we used our function f by writing f(x), to indicate the function\u0026rsquo;s variable. A notation identical to the one most used in calculus. Now, we just pass this equation to the dsolve function, indicating that it should be solved for f(x):\ndsolve(diff_eq, f(x)) \\( f(x) = C_{1} e^{x} \\)\nDo you agree with the result? Remember, the exponential function has the curious property of being equal to its own derivative ;-) This also holds for multiples of this function, hence the indication that there may be a constant \\(C_1\\) multiplying the function. Exactly the same notation that is usually used in calculus classes.\nTangent lines # By definition, the tangent line to a function \\(f(x)\\) at \\(x = x_0\\) is the line that passes through the point \\((x_0, f(x_0))\\) and has the same slope as the function at that point. Such a line is described by the equation\n$$ T(x) = f(x_0) + f'(x_0) (x - x_0) $$Let\u0026rsquo;s see this concept starting from the following very simple function:\n$$ f(x) = \\frac{1}{2} x^2 $$Let\u0026rsquo;s begin by creating the function with SymPy:\nfrom sympy import S f = S(\u0026#39;1/2\u0026#39;)*x**2 f \\( \\frac{1}{2} x^{2} \\)\nNow, let\u0026rsquo;s create the representation of its derivative:\ndf = diff(f, x) df \\( x \\)\nFollowing the definition, we can create a representation for the line T:\nx0 = Symbol(\u0026#39;x_0\u0026#39;) T = f.subs({x: x0}) + df.subs({x: x0}) * (x - x0) T \\(\\dfrac{x_{0}^{2}}{2} + x_{0} (x - x_{0}) \\)\nSo, if we want the tangent line to our function f at \\(x_0 = 1\\), all we need is:\nT.subs({x0: 1}) \\( x - \\dfrac{1}{2} \\)\nBy definition, this line has the same value and the same slope as the function \\(f(x)\\) at \\(x = 1\\). Let\u0026rsquo;s check:\n# checking that T(x0) == f(x0) T.subs({x0: 1}).subs({x: 1}) == f.subs({x: 1}) True # checking that the slope of T is the same as the function\u0026#39;s at the desired point diff(T, x).subs({x0: 1}) == diff(f, x).subs({x: 1}) True It is easier to visualize with a graph. Look at the graph generated by the code below, showing the function and its tangent line at \\(x_0 = 1\\):\nfrom sympy.plotting import plot import matplotlib.pyplot as plt def move_sympy_plot_to_axes(sympy_plot, plt_ax): \u0026#34;\u0026#34;\u0026#34;Moves a SymPy plot to a Matplotlib axes Adapted from: https://stackoverflow.com/a/46813804/8706250 Parameters ---------- sympy_plot : SymPy plot plt_ax : Matplotlib axes \u0026#34;\u0026#34;\u0026#34; backend = sympy_plot.backend(sympy_plot) backend.ax = plt_ax backend._process_series(backend.parent._series, plt_ax, backend.parent) backend.ax.spines[\u0026#39;right\u0026#39;].set_color(\u0026#39;none\u0026#39;) backend.ax.spines[\u0026#39;bottom\u0026#39;].set_position(\u0026#39;zero\u0026#39;) backend.ax.spines[\u0026#39;top\u0026#39;].set_color(\u0026#39;none\u0026#39;) plt.close(backend.fig) fig, ax = plt.subplots(figsize=(8, 6)) p1 = plot(f, (x, -3, 3), show=False, label=r\u0026#39;$f(x) = \\frac{1}{2}x^2$\u0026#39;) p2 = plot(x-S(\u0026#39;1/2\u0026#39;), (x, -3, 3), show=False, label=r\u0026#34;$f\u0026#39;(x) = x - \\frac{1}{2}$\u0026#34;) move_sympy_plot_to_axes(p1, ax) move_sympy_plot_to_axes(p2, ax) ax.scatter(1, 1/2, color=\u0026#39;red\u0026#39;, label=\u0026#39;Point (1, 1/2)\u0026#39;) ax.set_title(\u0026#39;Derivative as the slope of the tangent line at a point \\n\u0026#39;) ax.legend() plt.show() Conclusion # Did that bring back the calculus classes? We saw how simple it is to work with derivatives in SymPy, and we will use this simplicity in many of the future articles on the site.\nSee you next time.\n","date":"2022-03-20","externalUrl":null,"permalink":"/2022/03/calculus-python-sympy-derivatives/","section":"Blog","summary":"The start of a calculus course tends to feel abstract because of the concept of a limit. But it soon becomes more tangible with the introduction of the derivative, which has many obvious practical applications. In this article, we will see how to work with derivatives using SymPy, a Python package.","title":"Calculus with Python and SymPy - Derivatives","type":"posts"},{"content":" The concept of a limit is the most important one in the study of calculus. Many other concepts and ideas are built on the definition of a limit.\nIn this article, we will see how to find the limit of functions using SymPy and how this package can be a teaching aid in the study of calculus.\nInfinity in SymPy # Before getting into limits themselves, let\u0026rsquo;s see how the package represents infinity, since limits tending to infinity, or infinity as the answer to a calculus exercise, come up all the time.\nAs we have seen since the first article of this series on SymPy, it is a symbolic computer algebra system. So we need to know the internal representation of the concept of infinity:\n# setup for better outputs in the article, can be ignored from sympy import init_printing init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.05, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=10) from sympy import oo type(oo) sympy.core.numbers.Infinity Yes, that is two lowercase \u0026ldquo;o\u0026rdquo; letters :-)\nWe can do some simple operations to check whether the results are what we expect based on our mathematical knowledge. Let\u0026rsquo;s see:\noo + 1 \\( \\infty \\)\n1000 \u0026lt; oo True 1 / oo 0 As we see in the last three examples, the results agree with what we expect mathematically. Thus, in a very simple way, we have a representation of a concept that is not trivial.\nNow let\u0026rsquo;s move on to limits.\nThe limit function # The definition of a limit is somewhat abstract, at least at first glance:\nThe function \\(f\\) approaches a limit \\(l\\) near \\(a\\) means: for every \\(\\epsilon \u0026gt; 0\\) there is some \\(\\delta \u0026gt; 0\\) such that, for all \\(x\\), if \\(0 \u0026lt; |x-a| \u0026lt; \\delta\\), then \\(|f(x) - l| \u0026lt; \\epsilon\\).\nWell\u0026hellip; that is quite a tangle of symbols, and it is the starting point of many calculus concepts. But, for this article and for many discussions, we can start from a simpler, more intuitive definition of a limit:\nThe function \\(f\\) approaches a limit \\(l\\) near \\(a\\) if we can make \\(f(x)\\) as close as desired to \\(l\\) by taking \\(x\\) sufficiently close to, but not equal to, \\(a\\).\nIt may still sound abstract, but we will see several examples that should make it easier to understand.\nLet\u0026rsquo;s start with one of the best-known limits, one of the many ways of representing Euler\u0026rsquo;s number:\n$$ e \\equiv \\lim_{x \\to \\infty} \\left( 1 + \\frac{1}{x} \\right)^x $$Let\u0026rsquo;s check whether SymPy returns e for this limit:\nfrom sympy import limit, Symbol x = Symbol(\u0026#39;x\u0026#39;) limit( (1 + 1 / x)**x, x, oo) e As we can see, the result is the representation of Euler\u0026rsquo;s number. To understand it better, let\u0026rsquo;s look at the following plot:\nfrom sympy.plotting import plot from sympy import E import matplotlib.pyplot as plt def move_sympy_plot_to_axes(sympy_plot, plt_ax): \u0026#34;\u0026#34;\u0026#34;Moves a SymPy plot to a Matplotlib axes Adapted from: https://stackoverflow.com/a/46813804/8706250 Parameters ---------- sympy_plot : SymPy plot plt_ax : Matplotlib axes \u0026#34;\u0026#34;\u0026#34; backend = sympy_plot.backend(sympy_plot) backend.ax = plt_ax backend._process_series(backend.parent._series, plt_ax, backend.parent) backend.ax.spines[\u0026#39;right\u0026#39;].set_color(\u0026#39;none\u0026#39;) backend.ax.spines[\u0026#39;bottom\u0026#39;].set_position(\u0026#39;zero\u0026#39;) backend.ax.spines[\u0026#39;top\u0026#39;].set_color(\u0026#39;none\u0026#39;) plt.close(backend.fig) fig, ax = plt.subplots(figsize=(8, 6)) p1 = plot((1 + 1/x)**x, (x, 0, 50), show=False, label=r\u0026#39;$\\left( 1 + \\frac{1}{x} \\right)^x$\u0026#39;) p2 = plot(E, (x, 0, 50), show=False, label=\u0026#34;Euler\u0026#39;s number\u0026#34;) move_sympy_plot_to_axes(p1, ax) move_sympy_plot_to_axes(p2, ax) ax.get_lines()[1].set(color=\u0026#39;red\u0026#39;, linestyle=\u0026#39;--\u0026#39;) ax.set_ylim(0, 3) ax.legend(loc=\u0026#39;lower right\u0026#39;) plt.show() See in the plot how the blue curve approaches the dashed red line, which represents Euler\u0026rsquo;s number, as x becomes larger and larger. That is, in the limit as x tends to infinity, the function approaches the value of Euler\u0026rsquo;s number.\nOne-sided limits # Let\u0026rsquo;s look at the plot of the following function f:\n$$ f(x) = \\frac{1}{x} $$from sympy.plotting import plot x = Symbol(\u0026#39;x\u0026#39;) f = 1 / x plot(f, (x, -10, 10), ylim=(-10, 10)); As we can see, the way we approach x=0 affects the limit. This is a case of one-sided limits. If we approach zero from the right, usually denoted by the symbol +, we get one limit. Approaching from the left, denoted by the symbol -, we get another value for the limit. In SymPy, we can indicate the direction of approach with the dir parameter (for direction) of the limit function:\nlimit(f, x, 0, dir=\u0026#39;+\u0026#39;) \\( \\infty \\)\nlimit(f, x, 0, dir=\u0026#39;-\u0026#39;) \\( - \\infty \\)\nThis parameter only needs to be given in ambiguous cases. For limits tending to infinity (negative and positive), it is not necessary, as we see in the plot:\nlimit(f, x, -oo), limit(f, x, oo) \\( (0, \\ 0) \\)\nTwo oscillating functions # Let\u0026rsquo;s see a few more examples that usually appear in calculus textbooks. Starting with the function g below:\n$$ g(x) = \\frac{\\sin(x)}{x} $$from sympy import sin, exp g = sin(x) / x plot(g, (x, -30, 30)); Let\u0026rsquo;s look at two limits of this function:\nlimit(g, x, 0) 1 limit(g, x, oo) 0 That the limit as x tends to zero is 1 is quite evident from the plot. The limit as x tends to infinity becomes more evident when we widen the x axis range a bit, which is why I specified the range from -30 to 30 in the code. See how the function oscillates between positive and negative values, but the amplitude of the oscillation gets smaller and smaller as the values of x become more positive or more negative. Thus, in the limit as x tends to infinity, whether positive or negative, the function approaches zero.\nThe function h defined below can be analyzed similarly:\n$$ h(x) = \\frac{\\sin(x)^2}{x} $$h = sin(x)**2 / x plot(h, (x, -30, 30)); limit(h, x, 0) 0 limit(h, x, oo) 0 The limit as x tends to zero is evident in the plot. The limit as x tends to infinity can be analyzed like the previous function g. The difference is that the amplitude decreases during the oscillation, tending to zero, but the sign of the function does not change.\nWhich function dominates? # Let\u0026rsquo;s look at the function j defined below:\n$$ j(x) = \\frac{\\exp(x)}{x^{100}} $$Let\u0026rsquo;s start with the plot:\n# only to turn off the numerical error warning import warnings warnings.filterwarnings(\u0026#39;ignore\u0026#39;, category=RuntimeWarning) j = exp(x) / (x**100) plot(j); See how the vertical scale of the plot extends to values on the order of \\(10^{138}\\). This makes it hard to analyze and understand the limit values we want to evaluate. Let\u0026rsquo;s make some plots with more suitable scales. We start by restricting the ranges of x and y:\nplot(j, (x, -2, 2), ylim=(0, 10)); Now we have a slightly better idea of the behavior of the plot. We see that near \\(x = \\pm 1\\) the function grows very quickly. Let\u0026rsquo;s check the results of the following four limits, comparing them with the plot shown:\nlimit(j, x, 0) \\( \\infty \\)\nlimit(j, x, 1) e limit(j, x, -oo) 0 limit(j, x, oo) \\( \\infty \\)\nThe first three limits are evident from the plot and from the expression of the function, don\u0026rsquo;t you agree? Near zero, the function tends to infinity; near 1 the function tends to e (substitute 1 into the function if you have trouble seeing it in the plot); and, when x tends to negative infinity, the function tends to zero, consistent with the plot showing the curve close to the x axis. What seems strange is the last limit, when x tends to positive infinity. After all, the plot shows the curve hugging the x axis, which leads us to believe the result should also be zero.\nTo understand it, let\u0026rsquo;s start by plotting the numerator and denominator functions:\nh_numerator = exp(x) plot_h_numerator = plot(h_numerator, (x, -2, 2), ylim=(0, 100), show=False, label=\u0026#39;Numerator\u0026#39;) h_denominator = x**100 plot_h_denominator = plot(h_denominator, (x, -2, 2), ylim=(0, 100), show=False, label=\u0026#39;Denominator\u0026#39;) fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, figsize=(8, 6)) move_sympy_plot_to_axes(plot_h_numerator, ax1) move_sympy_plot_to_axes(plot_h_denominator, ax2) ax1.set_title(\u0026#39;Numerator - $\\exp(x)$ \\n\u0026#39;) ax2.set_title(\u0026#39;Denominator - $x^{100}$ \\n\u0026#39;) plt.show() In the range shown, the denominator function grows very quickly near \\(x \\pm 1\\). We also see that the exponential function in the numerator is increasing. Now, if the numerator function is increasing, it may happen that at some point it becomes larger than the denominator. But, before checking this numerically, let\u0026rsquo;s recall some usual discussions in the study of mathematical functions.\nWhen we evaluate functions that result from sums, differences, products or quotients of other functions of different types, or of different powers of the same type of function, it is usual to say that one function \u0026ldquo;dominates\u0026rdquo; the other. This means that when x tends to negative or positive infinity, the plot of the function will resemble that of the dominant function. Thus, we have the following rules of thumb, usually discussed and detailed in calculus courses:\nexponential functions dominate polynomial functions polynomial functions dominate logarithmic functions among exponential functions, larger bases dominate smaller bases among polynomial functions, higher orders dominate lower orders Therefore, in the case at hand, the exponential function in the numerator will dominate the polynomial function in the denominator. This means it makes sense for the function j to tend to infinity as x tends to infinity, since the exponential function is increasing. But the plot of j, at least in the window shown, does not show this happening; it seems to tend to zero\u0026hellip;\nWell, at some point the numerator must equal the denominator and then become larger than it. To find the point (or points) where this equality occurs, we just need to solve the following equation:\n$$ \\exp(x) - x^{100} = 0 $$We can use the solve function already presented in this series of articles:\nfrom sympy import solve solution = solve(exp(x) - x**100, x, numerical=False) [s.n() for s in solution] \\( 1.01015271985388, \\ -0.990147384359501, \\ 647.2775124394 \\)\nWe see that there are three values of x that satisfy the equation. Note that the last one is particularly high, well above the range shown in the previous plots. Let\u0026rsquo;s make a plot overlaying the numerator and denominator functions over a range of x values surrounding the value found:\nfig, ax = plt.subplots(figsize=(8, 6)) h_numerator = exp(x) plot_h_numerator = plot(h_numerator, (x, 600, 650), show=False) h_denominator = x**100 plot_h_denominator = plot(h_denominator, (x, 600, 650), show=False) p1 = plot_h_numerator p2 = plot_h_denominator move_sympy_plot_to_axes(p1, ax) move_sympy_plot_to_axes(p2, ax) ax.legend(loc=\u0026#39;upper left\u0026#39;) plt.show() Now we do see that the exponential function really dominates from x of about 647.3 on. Thus, we can redo the plot of the function j with a more suitable scale to visualize this fact:\nplot(j, (x, 640, 650), axis_center=(640, 0)); We see that, indeed, the function starts to increase and, in the limit as x tends to infinity, tends to infinity.\nWe can also check these changes in the behavior of the function by substituting some example values:\nexamples = (-1000, -2, -1, -0.9, -0.1, 0.1, 0.9, 1, 1.01, 1.1, 2, 100, 500, 600, 650, 1000) row_length = 35 print(f\u0026#39;x {\u0026#34; \u0026#34; * (row_length//2)} | {\u0026#34; \u0026#34; * (row_length//2)} j(x)\u0026#39;) print(\u0026#39;-\u0026#39; * 45) for number in examples: value = (exp(number) / number**100).n() spaces = str(row_length - len(str(number))) print(f\u0026#39;{number:\u0026lt;{spaces}}{value:.3e}\u0026#39;) x | j(x) --------------------------------------------- -1000 5.076e-735 -2 1.068e-31 -1 3.679e-1 -0.9 1.531e+4 -0.1 9.048e+99 0.1 1.105e+100 0.9 9.260e+4 1 2.718e+0 1.01 1.015e+0 1.1 2.180e-4 2 5.829e-30 100 2.688e-157 500 1.779e-53 600 5.775e-18 650 1.000e+1 1000 1.970e+134 Notice how the values really do move away from values near zero to large values as x increases. Interesting, right?\nThe Squeeze Theorem # To wrap up this article, we will see the so-called Squeeze Theorem, also known as the Sandwich Theorem. Let\u0026rsquo;s start with the formal statement:\nLet \\(I\\) be an interval having the point \\(a\\) as a limit point. And let g, f and h be functions defined on \\(I\\) except possibly at \\(a\\). Suppose that for every \\(x\\) in \\(I\\) not equal to \\(a\\) we have $$g(x) \\le f(x) \\le h(x)$$ and suppose also that $$ \\lim_{x \\to a} g(x) = \\lim_{x \\to a} h(x) = L $$ Then $$\\lim_{x \\to a} f(x) = L $$OK, very abstract. Let\u0026rsquo;s see an example, starting from the following function:\n$$ k(x) = x^2 \\sin \\left( \\frac{1}{x} \\right) $$Let\u0026rsquo;s start with the plot of the function:\nk = x**2 * sin(1/x) plot(k, (x, -0.25, 0.25)); Graphically, we see that the limit of this function as x tends to zero must be zero. Let\u0026rsquo;s see the result from SymPy:\nlimit(k, x, 0) 0 Expected result. But so what?\nIf you wanted to solve the exercise by hand, you would probably start with the following limit property:\n$$ \\lim_{x \\to a} (f(x) \\cdot g(x)) = \\lim_{x \\to a} f(x) \\cdot \\lim_{x \\to a} g(x) $$However, \\(\\lim\\limits_{x \\to 0} \\sin \\left( \\frac{1}{x} \\right)\\) does not exist. So another way of solving it must be adopted.\nFrom the very definition of the sine function, we have: \\(-1 \\le \\sin \\left( \\frac{1}{x} \\right) \\le 1\\). It follows, therefore, that \\(-x^2 \\le x^2 \\sin \\left( \\frac{1}{x} \\right) \\le x^2\\). Since \\(\\lim\\limits_{x \\to 0} -x^2 = \\lim\\limits_{x \\to 0} x^2 = 0\\), the squeeze theorem gives \\(\\lim\\limits_{x \\to 0} x^2 \\sin \\left( \\frac{1}{x} \\right) = 0\\).\nWe can use SymPy to visually check the meaning of what we have just seen:\nfig, ax = plt.subplots(figsize=(8, 6)) p1 = plot(k, (x, -0.25, 0.25), show=False, label=r\u0026#39;$x^2 \\sin \\left( \\frac{1}{x} \\right)$\u0026#39;) p2 = plot(-x**2, (x, -0.25, 0.25), show=False, label=\u0026#39;$-x^2$\u0026#39;) p3 = plot(x**2, (x, -0.25, 0.25), show=False, label=\u0026#39;$x^2$\u0026#39;) move_sympy_plot_to_axes(p1, ax) move_sympy_plot_to_axes(p2, ax) move_sympy_plot_to_axes(p3, ax) ax.legend(loc=\u0026#39;upper left\u0026#39;) plt.show() We see how the plot of our function of interest lies between the plots of the other functions. Hence the name \u0026ldquo;sandwich\u0026rdquo; :-)\nConclusion # That was a good calculus review, wasn\u0026rsquo;t it? We saw several examples that are usual in calculus classes and how using SymPy lets us enrich the discussion further, thanks to how easily it generates plots and obtains results that are not always quickly evident, leading to deeper discussions.\nSee you next time.\n","date":"2022-03-10","externalUrl":null,"permalink":"/2022/03/calculus-python-sympy-limits/","section":"Blog","summary":"The concept of a limit is the most important one in the study of calculus. Many other concepts and ideas are built on the definition of a limit. In this article, we will see how to find the limit of functions using SymPy and how this package can be a teaching aid in the study of calculus.","title":"Calculus with Python and SymPy: limits","type":"posts"},{"content":"","date":"2022-03-05","externalUrl":null,"permalink":"/tags/complex-numbers/","section":"tags","summary":"","title":"complex numbers","type":"tags"},{"content":" Most likely your first contact with complex numbers was in high school, in a very quick and abstract way. However, it is a fascinating topic with many practical applications, ranging from vector graphics and sound frequency analysis to the study of fractals. In this article, we will see how the Python language handles complex numbers natively and how the SymPy package extends the power of the language.\nA (very) brief review of complex numbers # This article does not even remotely intend to give a thorough treatment of the theory of complex numbers. But in case the reader needs a little help because it has been a long time since they saw the subject, here is a brief review.\nA complex number is an element of a number system that contains the real numbers and an element denoted i, called the imaginary unit, which satisfies the equation \\(i^2 = -1\\). In engineering and computing, it is common to use the letter j for the imaginary unit. In fact, this is the form used natively by Python.\nComplex numbers allow every polynomial equation to have solutions. That was probably the starting point of the study of complex numbers for many who saw this topic in high school. For example, equations such as \\((x + 1)^2 + 9 = 0\\) were once considered to have no solution. However, they have no real solution, since they are usually represented on a real plane. Let\u0026rsquo;s use a bit of what we have already seen in another article to plot this equation on the real plane:\nfrom sympy import Symbol, solve, init_printing from sympy.plotting import plot # settings for better outputs in the article, can be ignored init_printing(scale=1.25, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;, fontsize=12) x = Symbol(\u0026#39;x\u0026#39;) example = (x + 1)**2 + 9 plot(example, axis_center=(0, 0)) You learned that the plot above shows there are no real roots for the equation in question, since there is no intersection with the x axis. But this equation does have complex solutions: \\(-1 +3i\\) and \\(-1 -3i\\). We can check with SymPy itself:\nsolve(example) \\(\\displaystyle \\left[ -1 - 3 i, \\ -1 + 3 i\\right]\\)\nThere is certainly much more to say about complex numbers, but I prefer to show it little by little. Before we continue exploring the subject with SymPy, let\u0026rsquo;s see how Python handles complex numbers natively. Along the way, we will keep reviewing the main aspects of complex numbers.\nComplex numbers in Python # As already mentioned, the language uses the letter j to represent complex numbers:\nz = 4 + 3j type(z) complex It is useful to split a complex number into two parts: the real part and the imaginary part, the latter being the real number that multiplies the imaginary unit. We can get these parts as attributes of our object z:\nz.real \\(\\displaystyle 4.0\\)\nz.imag \\(\\displaystyle 3.0\\)\nAnother useful concept in the study of complex numbers is the complex conjugate. The conjugate of a given complex number has the same real part, but the sign of the imaginary part is flipped. The conjugate can be obtained with the conjugate method:\nz.conjugate() (4-3j) Graphical representations of complex numbers # The first representation usually presented in the study of complex numbers is the Argand-Gauss plane. In this plane, the horizontal axis represents the real part of the complex number and the vertical axis the imaginary part. Thus, given a complex number in algebraic form \\(z = a + bi\\), we get a point (a, b) in this plane.\nNow, if we have a point in a plane we can define a vector from the origin of the coordinate system to the point. Let\u0026rsquo;s see, with the help of Matplotlib:\nimport matplotlib.pyplot as plt # to standardize the style of the plots in the article: params = { \u0026#39;axes.labelsize\u0026#39;: 12, \u0026#39;axes.titlesize\u0026#39;: 12, \u0026#39;axes.spines.top\u0026#39;: False, \u0026#39;axes.spines.right\u0026#39;: False, \u0026#39;xtick.labelsize\u0026#39;: 12, \u0026#39;ytick.labelsize\u0026#39;: 12, \u0026#39;figure.autolayout\u0026#39;: True, \u0026#39;figure.facecolor\u0026#39;: \u0026#39;white\u0026#39;, \u0026#39;figure.titlesize\u0026#39;: 16, \u0026#39;figure.figsize\u0026#39;: (12, 8), \u0026#39;legend.shadow\u0026#39;: False, \u0026#39;legend.fontsize\u0026#39;: 10, \u0026#39;lines.linewidth\u0026#39;: 2.0, } plt.rcParams.update(params) fig, ax = plt.subplots(figsize=(6, 6)) # point ax.scatter(z.real, z.imag, s=100, color=\u0026#39;red\u0026#39;) # vector ax.quiver(0, 0, z.real, z.imag, units=\u0026#39;xy\u0026#39;, angles=\u0026#39;xy\u0026#39;, scale=1) # labeling the plot properly ax.set_xlabel(\u0026#39;Re\u0026#39;) ax.set_ylabel(\u0026#39;Im\u0026#39;) fig.suptitle(\u0026#39;Argand-Gauss plane\u0026#39;, color=\u0026#39;dimgray\u0026#39;) ax.set_title(\u0026#39;Complex number: $z = 4 + 3i$\u0026#39;) plt.show() Being a vector, by definition it must have a magnitude, or modulus (the length of the vector), and it can be decomposed into its component vectors. Let\u0026rsquo;s improve our representation even further:\nimport math from matplotlib.ticker import MultipleLocator from matplotlib.patches import Arc fig, ax = plt.subplots() # building our coordinate system and cleaning up the plot ax.spines[\u0026#39;top\u0026#39;].set_color(\u0026#39;none\u0026#39;) ax.spines[\u0026#39;bottom\u0026#39;].set_position(\u0026#39;center\u0026#39;) ax.spines[\u0026#39;left\u0026#39;].set_position(\u0026#39;center\u0026#39;) ax.spines[\u0026#39;right\u0026#39;].set_color(\u0026#39;none\u0026#39;) ax.xaxis.set_ticks_position(\u0026#39;bottom\u0026#39;) ax.yaxis.set_ticks_position(\u0026#39;left\u0026#39;) ax.xaxis.set_major_locator(MultipleLocator(1)) ax.yaxis.set_major_locator(MultipleLocator(1)) ax.grid(which=\u0026#39;both\u0026#39;, color=\u0026#39;grey\u0026#39;, linewidth=1, linestyle=\u0026#39;-\u0026#39;, alpha=0.2) # point and vector ax.scatter(z.real, z.imag, s=100, color=\u0026#39;red\u0026#39;) ax.quiver(0, 0, z.real, z.imag, units=\u0026#39;xy\u0026#39;, angles=\u0026#39;xy\u0026#39;, scale=1) # circle circle = plt.Circle((0, 0), 5, color=\u0026#39;blue\u0026#39;, fill=False) ax.set_aspect(\u0026#39;equal\u0026#39;, adjustable=\u0026#39;datalim\u0026#39;) ax.add_patch(circle) # component vectors ax.quiver(0, 0, z.real, 0, units=\u0026#39;xy\u0026#39;, angles=\u0026#39;xy\u0026#39;, scale=1, color=\u0026#39;red\u0026#39;, width=0.075, alpha=0.25) ax.quiver(0, 0, 0, z.imag, units=\u0026#39;xy\u0026#39;, angles=\u0026#39;xy\u0026#39;, scale=1, color=\u0026#39;red\u0026#39;, width=0.075, alpha=0.25) # lines to highlight the triangles ax.hlines(z.imag, 0, z.real, linestyle=\u0026#39;--\u0026#39;, color=\u0026#39;purple\u0026#39;) ax.vlines(z.real, 0, z.imag, linestyle=\u0026#39;--\u0026#39;, color=\u0026#39;purple\u0026#39;) # showing the angle angle = math.degrees(math.asin(z.imag/abs(z))) # thanks, Pythagoras! arc = Arc(xy=(0, 0), width=2, height=2, theta1=0, theta2=angle, linewidth=2) ax.add_patch(arc) ax.text(1.1, 0.4, f\u0026#39;{angle:.1f}°\u0026#39;, fontsize=14) # labeling the plot properly ax.set_xlabel(\u0026#39;Re\u0026#39;, loc=\u0026#39;right\u0026#39;) ax.set_ylabel(\u0026#39;Im\u0026#39;, rotation=\u0026#39;horizontal\u0026#39;, ha=\u0026#39;right\u0026#39;, y=0.96, labelpad=-50) fig.suptitle(\u0026#39;Argand-Gauss plane\u0026#39;, color=\u0026#39;dimgray\u0026#39;) ax.set_title(\u0026#39;Complex number: $z = 4 + 3i$\u0026#39;) plt.show() In our representation, we see that the magnitude of the vector is 5 (see how the vector rotated around the origin traces a circle of radius 5, shown in blue). We can reach the same conclusion by noticing that a right triangle is formed, as the dashed lines in the plot try to highlight. It is clear that the vector is the hypotenuse of the triangle and the value can be found with the Pythagorean theorem.\nPython\u0026rsquo;s abs function can be used to get the value:\nabs(z) \\(\\displaystyle 5.0\\)\nWe can verify that it is the same value obtained by the Pythagorean theorem from the lengths of the component vectors:\nabs(z) == math.sqrt(z.real**2 + z.imag**2) True Another aspect related to the plot that is worth recalling: the complex number can be represented by a point. Python knows this:\ncomplex(4, 3) == 4 + 3j True That is, instead of writing out the algebraic form, we just pass the real part and the imaginary part to the complex function as if they were a coordinate pair.\nGo back to the previous plot. Notice that I highlighted the angle formed by the vector with the horizontal axis. Well, if we have an angle and a triangular representation, we can use trigonometric functions, as we already saw in another article of this SymPy series.\nThe complex number can therefore be represented in what we call the trigonometric form:\n$$ z = \\rho (\\cos \\theta + i \\sin \\theta) $$where \\(\\rho\\) represents the modulus of the vector.\nNow, it would be interesting if we could go from the algebraic form to the trigonometric form with little effort. And, of course, it is possible. See, mathematically it is quite simple to go from one form to the other, just apply a bit of Pythagoras (by the way, see the code that generated the plot for a demonstration of this and for an easter egg). What we are after here is a way to do this with code.\nFirst, let\u0026rsquo;s see how the math library, which already comes with Python, handles the following situation:\nmath.sqrt(-4) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[12], line 1 ----\u0026gt; 1 math.sqrt(-4) ValueError: math domain error OK, as expected, it raised an error because it does not know how to compute the square root of a negative number. But we know it is possible to compute, the result will just be complex and not real. So there is also the standard library cmath to handle situations where complex numbers may be involved. Let\u0026rsquo;s see:\nimport cmath cmath.sqrt(-4) 2j Bingo! It computed correctly, giving the complex result.\ncmath has a method, polar, that gives us exactly what we are after:\ncmath.polar(z) \\(\\displaystyle \\left( 5.0, \\ 0.643501108793284\\right)\\)\nLet\u0026rsquo;s understand the result piece by piece. First, the name of the method: polar. Another name for the trigonometric form is the polar form. That is because a complex number can be represented in a polar coordinate system. Mathematically, if something can be represented by a point in a plane with a given distance from a reference point and a given angle against a reference direction, then it can be represented in a polar coordinate system. That is exactly our situation. If that is not clear, later on we will see a representation.\nWhat did the method return? Well, the distance to the reference point (our modulus) and the value of the angle. It happens that the value of the angle is in radians and not in degrees as in the plot. If we convert it, we will see that they are equal:\nmath.degrees(cmath.polar(z)[1]) \\(\\displaystyle 36.869897645844\\)\nThe code above can be simplified if we know that the angle is also called the phase (phase) and, if we round the result to the first decimal place, we get exactly the same angle as in the plot:\nround(math.degrees(cmath.phase(z)), 1) \\(\\displaystyle 36.9\\)\nIf you are still not convinced, let\u0026rsquo;s work out the trigonometric/polar form in full and observe that we get the algebraic form as the result:\nabs(z) * (math.cos(cmath.phase(z)) + math.sin(cmath.phase(z))*1j) (4+3j) After all, it is just applying the modulus and the angle:\ncmath.polar(z) == (abs(z), cmath.phase(z)) True Convinced?\nTo finish this section, let\u0026rsquo;s see the representation of our example complex number in a polar coordinate system:\nfig, ax = plt.subplots(subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;polar\u0026#39;}, figsize=(6, 6)) ax.plot(cmath.phase(z), abs(z), marker=\u0026#39;o\u0026#39;, markersize=15, color=\u0026#39;red\u0026#39;) plt.show() Note that so far we have used only what plain Python gives us. What will SymPy add?\nComplex numbers with SymPy # Well, let\u0026rsquo;s start by importing a few things we will use:\nfrom sympy import I, re, im, Abs, arg, conjugate, solve, Symbol SymPy\u0026rsquo;s representation of the imaginary unit is I:\nI \\(\\displaystyle i\\)\nIt is an internal representation of SymPy itself:\ntype(I) sympy.core.numbers.ImaginaryUnit Since SymPy is for symbolic algebra, we see that the product of two imaginary units is indeed -1:\nI * I \\(\\displaystyle -1\\)\nLet\u0026rsquo;s recreate the complex number we have been using as an example since the beginning of the article, but now as a SymPy symbol:\nz = 4 + 3 * I z \\(\\displaystyle 4 + 3 i\\)\nInstead of treating the imaginary part and the real part as attributes of the object, SymPy has functions to which we pass a complex number and which return the parts. They are the functions re and im for the real and imaginary parts, respectively:\nre(z) \\(\\displaystyle 4\\)\nim(z) \\(\\displaystyle 3\\)\nLikewise, there is Abs to get the modulus, and conjugate to get the conjugate:\nAbs(z) \\(\\displaystyle 5\\)\nconjugate(z) \\(\\displaystyle 4 - 3 i\\)\nWe saw earlier that a complex number can be represented in polar coordinates from its modulus and its phase, the angle it makes with a reference axis. Another name for this angle is the argument. So there is the function arg to get the argument of a complex number:\narg(z) \\(\\displaystyle \\operatorname{atan}{\\left(\\frac{3}{4} \\right)}\\)\nSee the difference of a symbolic library. SymPy effectively returned that the angle is the one whose tangent is 3/4. Go back to the plot we made earlier and you will see that it really is the angle with tangent 3/4.\nIf you want the explicit value, use the n method as shown in the first article of this series to get a numerical approximation:\narg(z).n() \\(\\displaystyle 0.643501108793284\\)\nThe result comes out in radians. To get it in degrees and represent it with the same number of significant digits shown earlier in the plot, we do:\nfrom sympy import deg deg(arg(z)).n(3) \\(\\displaystyle 36.9\\)\nNow, let\u0026rsquo;s see the real power of a symbolic library.\nEuler\u0026rsquo;s formula # In a recent article about trigonometric functions, we saw that these can be written in terms of exponential functions with the so-called Euler\u0026rsquo;s formula:\n$$ e^{i \\theta} = \\cos(\\theta) + i \\sin(\\theta) $$Let\u0026rsquo;s see how SymPy recognizes this equality. First, let\u0026rsquo;s represent the left-hand side of the equality:\nfrom sympy import exp, sin, cos, symbols # SymPy recognizes LaTeX symbols :-) rho, theta = symbols(r\u0026#39;\\rho \\theta\u0026#39;, real=True) exp(I * theta) \\(\\displaystyle e^{i \\theta}\\)\nNow, let\u0026rsquo;s use the expand method, passing the information that it should consider complex numbers:\nexp(I * theta).expand(complex=True) \\(\\displaystyle i \\sin{\\left(\\theta \\right)} + \\cos{\\left(\\theta \\right)}\\)\nNote that this is exactly the right-hand side of the equality.\nAnd the reverse? Given the right-hand side of the equation, we can ask for it to be simplified:\n(cos(theta) + I * sin(theta)).simplify() \\(\\displaystyle e^{i \\theta}\\)\nWe got exactly the left-hand side of the equality.\nWe can multiply both sides of the equality by any value. Let\u0026rsquo;s multiply by the symbol \\(\\rho\\), which we are using to represent the modulus:\n$$ \\rho e^{i \\theta} = \\rho (\\cos(\\theta) + i \\sin(\\theta)) $$Well, the right-hand side has become the polar form representation of a generic complex number, which we already saw earlier.\nLet\u0026rsquo;s then try to obtain the algebraic form of a complex number from its exponential form. We will use the complex number \\(4 + 3i\\), our example since the beginning. We already know the modulus and the argument for this complex number, so we can substitute the values, obtaining an expression for the left-hand side of the equality:\nfrom sympy import atan expr = (rho * exp(I * theta)).subs({rho: 5, theta: atan(\u0026#39;3/4\u0026#39;)}) expr \\(\\displaystyle 5 e^{i \\operatorname{atan}{\\left(\\frac{3}{4} \\right)}}\\)\nSymPy has the as_real_imag method which, as the name suggests, tries to represent the object as a tuple (real part, imaginary part). That is, as a point in the plane, as seen earlier in the article. Let\u0026rsquo;s see the result of this method:\nexpr.as_real_imag() \\(\\displaystyle \\left( 4, \\ 3\\right)\\)\nExactly as expected! That is, SymPy takes the ability to work with complex expressions to another level. With plain Python, we need to know the relations and build them manually. With SymPy, the transformations can be done easily, with simple method calls.\nConclusion, more complex numbers, and more SymPy # Complex numbers are not such a complex subject ;-). With the right tools and visualizations the subject becomes more tangible.\nSee you next time.\n","date":"2022-03-05","externalUrl":null,"permalink":"/2022/03/complex-numbers-python-sympy/","section":"Blog","summary":"Complex numbers are fascinating and have many practical applications. In this article, we will see how Python handles complex numbers natively and how the SymPy package extends the power of the language.","title":"Complex Numbers with Python and SymPy","type":"posts"},{"content":"","date":"2022-03-05","externalUrl":null,"permalink":"/tags/complexos/","section":"tags","summary":"","title":"complexos","type":"tags"},{"content":"Linux Mint is a distribution based on Ubuntu whose main desktop environment is Cinnamon. Its look is more familiar to people migrating from Windows, and it meets the needs of all kinds of users.\nIn this video (in Portuguese), we will see how to install Linux Mint. Learn how to download it, create the bootable USB drive, and go through the installation process. We will see how to encrypt the installation partition and the personal data partition, and the first recommended settings right after the installation.\nAdditional links # Linux Mint website\nVideo from the Explaining Computer channel about the BIOS, showing how to change the boot order so the computer starts from the USB drive.\nLink to download Etcher, a program for creating the bootable USB drive.\nLinux Mint and Programming # Here at Ciência Programada we write a lot about programming. We have already made a video (in Portuguese) showing how to install the Anaconda distribution on Linux Mint to program with Python:\nLink to the article about Anaconda.\nWe also have a video and an article showing how to create virtual environments for Python projects on Linux Mint.\n","date":"2022-03-01","externalUrl":null,"permalink":"/2022/03/how-to-install-linux-mint/","section":"Blog","summary":"Linux Mint is a distribution based on Ubuntu whose main desktop environment is Cinnamon. Its look is more familiar to people migrating from Windows, and it meets the needs of all kinds of users.","title":"How to Install Linux Mint","type":"posts"},{"content":"","date":"2022-02-07","externalUrl":null,"permalink":"/tags/matplotlib/","section":"tags","summary":"","title":"Matplotlib","type":"tags"},{"content":"","date":"2022-02-07","externalUrl":null,"permalink":"/tags/mendeleev/","section":"tags","summary":"","title":"mendeleev","type":"tags"},{"content":"","date":"2022-02-07","externalUrl":null,"permalink":"/tags/molmass/","section":"tags","summary":"","title":"molmass","type":"tags"},{"content":" Top 3 Python Libraries for Chemistry # Chemistry students and professionals need to perform many calculations every day. Such calculations are often complex because of the large amount of data and the approximations involved. Computational tools make these tasks easier.\nIn this article, we explore 3 chemistry libraries available for the Python language, showing how they are useful in our routine.\nMolmass # We start our Top 3 with Molmass, a Python package that calculates molar mass in a very simple way. Instead of tediously calculating the molar masses of compounds by adding up those of their constituent elements, you just pass the formula, as we will see.\nInstallation can be done via pip install molmass. And, to import:\nfrom molmass import Formula Let\u0026rsquo;s start with a simple example: beer! More specifically, the ethanol found in beers.\nRepresentation of the ethanol molecule We can easily calculate the molar mass of ethanol with Molmass. To do so, we pass a string with the ethanol formula to the Formula class and request the mass attribute. The unit is g/mol, as noted in the comment:\nformula = Formula(\u0026#39;C2H5OH\u0026#39;) formula.mass # g/mol 46.068531 We can see that the package uses molar masses without rounding, which is excellent, since the use of approximations leads to error propagation.\nOh, and if you want to keep talking about beer, see how it has already given rise to a Python library in this article. But only after you finish this article :-)\nMolmass accepts different inputs for the same compound. Let\u0026rsquo;s pass another string, with the classic abbreviation of the ethyl radical in organic chemistry, Et, to obtain the molar mass of ethanol:\nformula = Formula(\u0026#39;EtOH\u0026#39;) formula.mass 46.068531 Note that formulas are case sensitive.\nTo get information on the elemental composition of the compound, we can use the composition method:\nformula.composition() (('C', 2, 24.02148, 0.5214292593788155), ('H', 6, 6.047646, 0.13127499116479316), ('O', 1, 15.999405, 0.34729574945639136)) We see that the composition method provides columns with the attributes element, number, relative_mass, and fraction, respectively.\nMendeleev # Another Python package that is very useful in our daily work is Mendeleev. It gives access to many properties of elements, ions, and isotopes.\nThe package can be installed via pip install mendeleev.\nPeriodic table # One of the coolest features of Mendeleev is the ability to import a periodic table with just one line of code. For that we need the vis dependency, which can be installed via pip install mendeleev[vis].\nOne way to visualize the periodic table is to use the periodic_table function from the mendeleev.vis module:\nfrom mendeleev.vis import periodic_table To see the default visualization of the periodic table, let\u0026rsquo;s call the imported function:\nperiodic_table() Properties of elements # Now from a more technical point of view. To access the data of an element, we just import it by its symbol. For example, consider chromium:\nfrom mendeleev import Cr To extract some information we can request an attribute, for example, boiling_point, which gives the boiling temperature in kelvin, as noted in the comment:\nCr.boiling_point # K 2945.0 An alternative way to access the data is through element, a method that returns a single instance or a tuple of such instances, depending on the arguments:\nfrom mendeleev import element The element method accepts the element name, the atomic symbol, or the atomic number. Let\u0026rsquo;s see an example in which we pass the atomic number of bromine to the element method and request the atomic_radius attribute, which gives the atomic radius in picometers, as noted in the comment:\nbromine = element(35) bromine.atomic_radius # pm 115.0 The element method also accepts a list or tuple. Look at this example, in which we pass a list to the element method and request the econf attribute, which gives the electron configuration of the elements:\nhydrogen, zinc, bromine = element([\u0026#39;Hydrogen\u0026#39;, \u0026#39;Zn\u0026#39;, 35]) hydrogen.econf, zinc.econf, bromine.econf ('1s', '[Ar] 3d10 4s2', '[Ar] 3d10 4s2 4p5') With Mendeleev we can also obtain the electron configuration of an ion from its element. Take the Fe3+ ion as an example. First, we pass the element name to the element method and request the ec attribute, which gives the electron configuration of the iron atom:\niron = element(\u0026#39;Iron\u0026#39;) iron.ec \u0026lt;ElectronicConfiguration(conf=\u0026quot;1s2 2s2 2p6 3s2 3p6 3d6 4s2\u0026quot;)\u0026gt; To obtain the electron configuration of the Fe3+ ion we use the ionize method to remove the 2 electrons of the outermost shell and not from the highest-energy subshell and 1 electron, now yes, from the highest-energy subshell, returning the following electron configuration:\niron.ec.ionize(3) \u0026lt;ElectronicConfiguration(conf=\u0026quot;1s2 2s2 2p6 3s2 3p6 3d5\u0026quot;)\u0026gt; There are some attributes such as isotopes, which returns a list of objects with the properties of each isotope: atomic_number, mass_number, mass, and abundance, respectively:\nsulfur = element(\u0026#39;S\u0026#39;) for iso in sulfur.isotopes: print(iso) 16 32 31.97207 0.944 16 33 32.97146 0.008 16 34 33.96787 0.048 16 36 35.96708 0.000 ionic_radii returns a list of objects with the attributes charge, coordination, crystal_radius in pm, and ionic_radius in pm:\ngold = element(\u0026#39;Au\u0026#39;) for ir in gold.ionic_radii: print(ir) charge= 1, coordination=VI , crystal_radius=151.000, ionic_radius=137.000 charge= 3, coordination=IVSQ , crystal_radius=82.000, ionic_radius=68.000 charge= 3, coordination=VI , crystal_radius=99.000, ionic_radius=85.000 charge= 5, coordination=VI , crystal_radius=71.000, ionic_radius=57.000 ionenergies returns a dictionary with ionization energies:\noxygen = element(\u0026#39;O\u0026#39;) oxygen.ionenergies {1: 13.618054, 2: 35.12111, 3: 54.93554, 4: 77.4135, 5: 113.8989, 6: 138.1189, 7: 739.32679, 8: 871.40985} Another attribute is oxistates, which returns a list of the most common oxidation states of a given element:\nmanganese = element(\u0026#39;Mn\u0026#39;) manganese.oxistates [7, 6, 4, 3, 2] The element method has several other attributes and/or methods that let us access many properties of the elements. Let\u0026rsquo;s look at all of those available on the manganese object we created:\nmanganese = element(\u0026#39;Mn\u0026#39;) [x for x in dir(manganese) if not x.startswith(\u0026#39;_\u0026#39;)] ['abundance_crust', 'abundance_sea', 'annotation', 'atomic_number', 'atomic_radius', 'atomic_radius_rahm', 'atomic_volume', 'atomic_weight', 'atomic_weight_uncertainty', 'block', 'boiling_point', 'c6', 'c6_gb', 'cas', 'covalent_radius', 'covalent_radius_bragg', 'covalent_radius_cordero', 'covalent_radius_pyykko', 'covalent_radius_pyykko_double', 'covalent_radius_pyykko_triple', 'cpk_color', 'density', 'description', 'dipole_polarizability', 'dipole_polarizability_unc', 'discoverers', 'discovery_location', 'discovery_year', 'ec', 'econf', 'electron_affinity', 'electronegativity', 'electronegativity_allen', 'electronegativity_allred_rochow', 'electronegativity_cottrell_sutton', 'electronegativity_ghosh', 'electronegativity_gordy', 'electronegativity_li_xue', 'electronegativity_martynov_batsanov', 'electronegativity_mulliken', 'electronegativity_nagle', 'electronegativity_pauling', 'electronegativity_sanderson', 'electronegativity_scales', 'electrons', 'electrophilicity', 'en_allen', 'en_ghosh', 'en_pauling', 'evaporation_heat', 'fusion_heat', 'gas_basicity', 'geochemical_class', 'glawe_number', 'goldschmidt_class', 'group', 'group_id', 'hardness', 'heat_of_formation', 'init_on_load', 'ionenergies', 'ionic_radii', 'is_monoisotopic', 'is_radioactive', 'isotopes', 'jmol_color', 'lattice_constant', 'lattice_structure', 'mass', 'mass_number', 'mass_str', 'melting_point', 'mendeleev_number', 'metadata', 'metallic_radius', 'metallic_radius_c12', 'molcas_gv_color', 'name', 'name_origin', 'neutrons', 'nvalence', 'oxides', 'oxistates', 'period', 'pettifor_number', 'proton_affinity', 'protons', 'registry', 'sconst', 'screening_constants', 'series', 'softness', 'sources', 'specific_heat', 'symbol', 'thermal_conductivity', 'uses', 'vdw_radius', 'vdw_radius_alvarez', 'vdw_radius_batsanov', 'vdw_radius_bondi', 'vdw_radius_dreiding', 'vdw_radius_mm3', 'vdw_radius_rt', 'vdw_radius_truhlar', 'vdw_radius_uff', 'zeff'] Properties of ions # mendeleev has the Ion class for working with ions instead of elements. Ions can be created from elements and charge information:\nfrom mendeleev.ion import Ion iron_3 = Ion(\u0026#34;Fe\u0026#34;, 3) Let\u0026rsquo;s access some properties of the ions:\niron_3.electrons 23 iron_3.name 'Iron 3+ ion' iron_3.unicode_ion_symbol() 'Fe³⁺' ChemPy # ChemPy is a Python package that can solve problems in analytical chemistry, physical chemistry, and inorganic chemistry.\nThe package can be installed via pip install chempy.\nBalancing equations # With ChemPy we can balance the stoichiometric coefficients of a reaction. To import, simply:\nfrom chempy import balance_stoichiometry Let\u0026rsquo;s imagine an example in which we need to balance the combustion reaction of acetylene, a fuel gas used in blowtorches.\nAcetylene is the fuel of some blowtorches $$ C_2H_2 + O_2 \\rightarrow CO_2 + H_2O $$To do so, we pass the reactants and the products to the balance_stoichiometry function:\nbalance_stoichiometry({\u0026#39;C2H2\u0026#39;, \u0026#39;O2\u0026#39;}, {\u0026#39;CO2\u0026#39;, \u0026#39;H2O\u0026#39;}) (OrderedDict([('C2H2', 2), ('O2', 5)]), OrderedDict([('CO2', 4), ('H2O', 2)])) We got two dictionaries, one for the reactants and one for the products, with the stoichiometric coefficients that balance the equation.\nLet\u0026rsquo;s now consider another example to show that we can obtain the balanced equation directly. We will balance the following reaction, which is used in space rocket boosters to help launch them:\n$$ NH_4ClO_4 + Al \\rightarrow Al_2O_3 + HCl + H_2O + N_2 $$reac, prod = balance_stoichiometry({\u0026#39;NH4ClO4\u0026#39;, \u0026#39;Al\u0026#39;}, {\u0026#39;Al2O3\u0026#39;, \u0026#39;HCl\u0026#39;, \u0026#39;H2O\u0026#39;, \u0026#39;N2\u0026#39;}) reac OrderedDict([('Al', 10), ('NH4ClO4', 6)]) prod OrderedDict([('Al2O3', 5), ('H2O', 9), ('HCl', 6), ('N2', 3)]) This returns an ordered dictionary; by calling the dict function we create a standard dictionary:\ndict(reac) {'Al': 10, 'NH4ClO4': 6} dict(prod) {'Al2O3': 5, 'H2O': 9, 'HCl': 6, 'N2': 3} We can build the balanced equation using the Reaction class:\nfrom chempy import Reaction reaction = Reaction(reac, prod) reaction 10 Al + 6 NH4ClO4 -\u0026gt; 5 Al2O3 + 9 H2O + 6 HCl + 3 N2 Redox systems # Among the most beautiful and interesting chemistry experiments are the reactions that change color. These color-change experiments can occur, for example, as a result of reduction and oxidation reactions.\nThe GIF below shows one of these reactions:\nReaction of potassium permanganate with hydrogen peroxide In the GIF example, the color change occurs because purple permanganate ions, MnO4-, receive electrons to form colorless manganese(II) ions, Mn+2. In addition, gas forms because hydrogen peroxide, H2O2, and the oxygen of MnO4- are converted into H2O and O2.\nIf you need to balance this redox reaction, we can also do it with ChemPy. Let\u0026rsquo;s obtain the balanced overall equation from the following half-reactions:\n$$ \\begin{align*} MnO_4^- + 8H^+ + 5e^- \u0026\\rightleftharpoons Mn^{+2} + 4H_2O \\\\ H_2O_2 \u0026\\rightleftharpoons O_2 + 2H^+ + 2e^- \\end{align*} $$To do so, we pass the reactants and products of the respective half-reactions to the Equilibrium class:\nfrom chempy import Equilibrium sr_red = Equilibrium({\u0026#39;MnO4-\u0026#39;: 1, \u0026#39;H+\u0026#39;: 8, \u0026#39;e-\u0026#39;: 5}, {\u0026#39;Mn+2\u0026#39;: 1, \u0026#39;H2O\u0026#39;: 4}) sr_ox = Equilibrium({\u0026#39;H2O2\u0026#39;: 1}, {\u0026#39;O2\u0026#39;: 1, \u0026#39;H+\u0026#39;:2, \u0026#39;e-\u0026#39;:2}) Now we need to eliminate the electrons and multiply each reaction by its respective coefficient to obtain the balanced overall equation:\ncoeff = Equilibrium.eliminate([sr_red, sr_ox], \u0026#39;e-\u0026#39;) coeff [2, 5] redox = sr_red * coeff[0] + sr_ox * coeff[1] redox 6 H+ + 5 H2O2 + 2 MnO4- -\u0026gt; 8 H2O + 2 Mn+2 + 5 O2 Chemical kinetics # Besides balancing, ChemPy is very useful in the study of chemical kinetics.\nThe Arrhenius equation gives the dependence of the rate constant on temperature as:\n$$ k = A \\exp \\left( -\\frac{E_a}{RT} \\right) $$Taking the natural logarithm of the Arrhenius equation and rearranging, we get the form of a linear equation:\n$$ \\ln k = - \\frac{E_a}{R}\\frac{1}{T} + \\ln A $$ChemPy lets us obtain the activation energy and the parameter A using the fit_arrhenius_equation function. Let\u0026rsquo;s start by importing the function:\nfrom chempy.kinetics.arrhenius import fit_arrhenius_equation Now we will pass a set of temperature values in kelvin, rate constants in L/(mol.s), and errors of the rate constant. To do so, we import the NumPy library in order to create the arrays and to later use the natural logarithm (log) and exponential (exp) functions:\nimport numpy as np temperatures = np.array((1125, 1053, 1001, 838)) # K k_values = np.array((11.59, 1.67, 0.380, 0.0011)) # L / (mol.s) k_errors = np.array((1E-2, 1E-2, 1E-3, 1E-5)) Passing the variables temperatures, k_values, and k_errors to the fit_arrhenius_equation function, we obtain the values of the activation energy and of the parameter A:\nfit = fit_arrhenius_equation(temperatures, k_values, kerr=k_errors) fit (array([5.13001705e+12, 2.51435472e+05]), array([[inf, inf], [inf, inf]])) Internally, fit_arrhenius_equation uses the curve_fit method from SciPy. The return value is two arrays. The first holds the parameter values; the second holds the covariances, when they can be computed.\nTo create the variables for the parameter A and the activation energy, we access the elements of the first array, which is quite simple. The first array is ([5.13001705e+12, 2.51435472e+05]), whose values can be accessed as follows:\nA = fit[0][0] activation_energy = fit[0][1] print(f\u0026#39;A = {A:.3e} L/(mol.s) \\nEa = {activation_energy:.3e} J/mol\u0026#39;) A = 5.130e+12 L/(mol.s) Ea = 2.514e+05 J/mol We can also obtain the activation energy and the parameter A from the graph of \\(\\ln k\\) vs \\(\\frac{1}{T}\\).\nimport matplotlib.pyplot as plt from scipy.stats import linregress # plot config params = { \u0026#39;lines.linewidth\u0026#39;: 2.0, \u0026#39;axes.facecolor\u0026#39;: \u0026#39;white\u0026#39;, \u0026#39;figure.facecolor\u0026#39;: \u0026#39;white\u0026#39;, \u0026#39;axes.labelsize\u0026#39;: 14, \u0026#39;axes.titlesize\u0026#39;: 16, \u0026#39;xtick.labelsize\u0026#39;: 12, \u0026#39;ytick.labelsize\u0026#39;: 12, \u0026#39;figure.autolayout\u0026#39;: True, \u0026#39;figure.titlesize\u0026#39;: 18, \u0026#39;figure.figsize\u0026#39;: (12, 6), \u0026#39;legend.shadow\u0026#39;: False, \u0026#39;legend.fontsize\u0026#39;: 14, } plt.rcParams.update(params) # data points fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2) ax1.errorbar(temperatures, k_values, yerr=k_errors, marker=\u0026#39;o\u0026#39;, linestyle=\u0026#39;\u0026#39;) ax2.scatter(1/temperatures, np.log(k_values), label=\u0026#39;Data\u0026#39;) # regression slope, intercept, r_value, p_value, std_err = linregress(1/temperatures, np.log(k_values)) regression = slope * 1/temperatures + intercept ax2.plot(1/temperatures, regression, color=\u0026#39;red\u0026#39;, alpha=0.5, label=r\u0026#39;$\\ln k={:.0f}\\frac{{1}}{{T}}+{:.2f}$\u0026#39;.format(slope,intercept)) ax2.legend() # visuals for ax in (ax1, ax2): ax.ticklabel_format(style=\u0026#39;sci\u0026#39;, axis=\u0026#39;x\u0026#39;, scilimits=(0,0)) ax.xaxis.major.formatter._useMathText = True ax.figure.canvas.draw() order_magnitude = ax.xaxis.get_offset_text().get_text().replace(\u0026#39;\\\\times\u0026#39;, \u0026#39;\u0026#39;) ax.xaxis.offsetText.set_visible(False) if ax == ax1: ax.set_xlabel(r\u0026#39;$T$ / \u0026#39; + order_magnitude + \u0026#39; $K$\u0026#39;, ) ax.set_ylabel(r\u0026#39;$k / \\frac{\\ell}{mol \\cdot s}$\u0026#39;) ax.set_title(\u0026#39;Exponential form\u0026#39;, color=\u0026#39;dimgray\u0026#39;) else: ax.set_xlabel(r\u0026#39;$\\frac{1}{T}$ / \u0026#39; + order_magnitude + \u0026#39; $K^{-1}$\u0026#39;, ) ax.set_ylabel(r\u0026#39;$\\ln k$\u0026#39;) ax.set_title(\u0026#39;Linearized form\u0026#39;, color=\u0026#39;dimgray\u0026#39;) plt.suptitle(\u0026#39;Plots for the Arrhenius equation\u0026#39;, color=\u0026#39;dimgray\u0026#39;) plt.show() If you want to see how the graph was generated with the Matplotlib library, see this repository.\nSystems at equilibrium # ChemPy also performs pH calculations. Imagine an example in which we need to find the pH of a 0.015 mol/L acetate solution.\n$$ \\begin{align*} H_3CCOOH \u0026\\rightleftharpoons H^+ + H_3CCOO^- \u0026\u0026 K_a = 10^{-4.756} \\\\ H_2O \u0026\\rightleftharpoons H^+ + OH^- \u0026\u0026 K_w = 10^{-14} \\\\ \\end{align*} $$Let\u0026rsquo;s start by creating a system with the equilibrium equations and their constants:\nfrom chempy.equilibria import EqSystem equation = \u0026#39;H3CCOOH = H+ + H3CCOO-; 10**-4.756\u0026#39; water_autoionization = \u0026#39;H2O = H+ + OH-; 10**-14/55.5\u0026#39; system = EqSystem.from_string(\u0026#39;\\n\u0026#39;.join((equation, water_autoionization))) For ChemPy to solve the system and give the concentrations of all species in solution, we need to pass the concentrations of the species that are known. The acetate concentration is 0.015 mol/L and the molecular concentration of water is 55.5 mol/L:\nfrom collections import defaultdict condition = defaultdict(float, {\u0026#39;H2O\u0026#39;: 55.5, \u0026#39;H3CCOO-\u0026#39;: 0.015}) concentrations, _, _ = system.root(condition) result = dict(zip(system.substances, concentrations)) result {'H+': 3.4177702795991742e-09, 'H2O': 55.499997074115974, 'H3CCOO-': 0.014997077533741783, 'H3CCOOH': 2.922466258210992e-06, 'OH-': 2.9258840281057227e-06} Passing the H+ concentration to the following function returns the pH:\ndef pH(hydronium_concentration): return -np.log10(hydronium_concentration) print(f\u0026#34;{pH(result[\u0026#39;H+\u0026#39;]):.2f}\u0026#34;) 8.47 We can also obtain the species diagram for acetic acid. For that, we need the mole fractions of acetate and acetic acid. Let\u0026rsquo;s create a function that takes all the species in the system and removes the species that will not be used, returning a dictionary of the mole fractions of the species of interest, acetate and acetic acid:\ndef mole_fractions(substances): for key in (\u0026#39;H+\u0026#39;, \u0026#39;OH-\u0026#39;, \u0026#39;H2O\u0026#39;): substances.pop(key, None) frac = dict() total = sum(substances.values()) for key, value in substances.items(): frac[key] = value / total return frac Now let\u0026rsquo;s create a dictionary to store the mole fractions of the species of interest and create the pH range. fractions was initially created as an empty list to later store the mole fraction values for each pH value.\nspecies = [item for item in system.substance_labels() if item not in (\u0026#39;H+\u0026#39;, \u0026#39;OH-\u0026#39;, \u0026#39;H2O\u0026#39;)] fractions = defaultdict(list, {k: [] for k in species}) pH_range = np.arange(0, 14.1, 0.1) for value in pH_range: condition = defaultdict(float, {\u0026#39;H2O\u0026#39;: 55.5, \u0026#39;H+\u0026#39;: 10**-value}) concentrations, _, _ = system.root(condition) result = dict(zip(system.substances, concentrations)) mf = mole_fractions(result) for key, value in zip(fractions.keys(), mf.values()): fractions[key].append(value) We can obtain the species diagram plot using the Matplotlib library. To do so, we create a figure, pass the axes, and label the plot:\nfig, ax = plt.subplots() for compound in fractions: ax.plot(pH_range, fractions[compound], label=f\u0026#39;${system.substances[compound].latex_name}$\u0026#39;) ax.set_xlabel(\u0026#39;pH\u0026#39;) ax.set_ylabel(r\u0026#39;$\\alpha$\u0026#39;) ax.grid(b=True, axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=1.5) ax.minorticks_on() ax.grid(b=True, axis=\u0026#39;both\u0026#39;, which=\u0026#39;minor\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=1.0) ax.set_axisbelow(True) ax.legend() ax.set_title(\u0026#39;Species diagram for acetic acid\u0026#39;, color=\u0026#39;dimgray\u0026#39;) plt.show() Conclusion # In this article, we saw some features of the Molmass, Mendeleev, and ChemPy libraries and how useful these tools are for the routine tasks of chemistry students and professionals. We saw that we can speed up simple calculations, such as molar mass, and even more complex ones, such as species concentration and kinetics.\nThis article is a collaboration between me, Helena Benevenuto, and Ciência Programada. Follow me on LinkedIn for more chemistry content (in Portuguese).\nShare this article on your networks and follow the Ciência Programada project to always stay up to date. See you next time.\n","date":"2022-02-07","externalUrl":null,"permalink":"/2022/02/top-3-python-libraries-chemistry/","section":"Blog","summary":"In this article, we explore 3 chemistry libraries available for the Python language, showing how they are useful in the daily routine of chemistry professionals and students.","title":"Top 3 Python Libraries for Chemistry","type":"posts"},{"content":" What do ocean waves, soap bubbles, and DNA have in common? That is what I show in this article, with the help of Python.\nNot everyone has fond memories of trigonometric functions from school. And even in science and engineering degrees, some people turn up their noses whenever they see sine, cosine, and tangent. In this article we will see how to use SymPy to make our lives easier when working with trigonometric functions (and their \u0026ldquo;cousins,\u0026rdquo; the hyperbolic ones!), and we will also illustrate some real applications of these functions.\nTrigonometric functions # Just as a reminder, trigonometric functions can be defined as ratios between two sides of a right triangle as a function of an angle or, more generally, as ratios of the coordinates of points on the unit circle, illustrated below:\nUnit circle with the (cosine, sine) coordinates of the main angles. Source: Wikipedia Let us start by importing the libraries we will use. The role of each one will be explained throughout the article.\nimport sympy from sympy import (sin, cos, tan, asin, acos, atan, sqrt, symbols, simplify, trigsimp, log, exp, expand, expand_trig, Symbol, rad, deg, sinh, cosh, tanh) from sympy.plotting import (plot, plot3d, plot3d_parametric_line, plot_implicit, plot3d_parametric_surface) import numpy as np import matplotlib.pyplot as plt from matplotlib.ticker import FuncFormatter, MultipleLocator # setting for better outputs in the article, can be ignored sympy.init_printing(scale=1.0, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;,) Note that we import the SymPy trigonometric functions directly. That way, we can call them directly, without having to write sympy.\u0026lt;function\u0026gt;.\nBy default, SymPy\u0026rsquo;s trigonometric functions take angles in radians. So, if we want the sine, cosine, and tangent of 30°, we must provide the value in radians, π/6, as shown in the unit circle figure at the beginning of the article:\nsin(sympy.pi / 6) \\(\\displaystyle \\frac{1}{2}\\)\ncos(sympy.pi / 6) \\(\\displaystyle \\frac{\\sqrt{3}}{2}\\)\ntan(sympy.pi / 6) \\(\\displaystyle \\frac{\\sqrt{3}}{3}\\)\nThe same line of reasoning applies to the other trigonometric functions: secant (sec), cosecant (csc), and cotangent (cot).\nIf you do not know the value in radians for a given angle, there is SymPy\u0026rsquo;s rad function:\nrad(30) \\(\\displaystyle \\frac{\\pi}{6}\\)\nsin(rad(30)) \\(\\displaystyle \\frac{1}{2}\\)\nThere is also the deg function, to get the value in degrees of an angle given in radians:\ndeg(sympy.pi / 6) \\(\\displaystyle 30\\)\nInverse functions\u0026hellip; and beware of floats # Inverse functions are also called arc functions. They are the famous arcsine, arccosine, arctangent\u0026hellip; These functions return the angle associated with a value. For example, we know that the sine of 30° is 1/2. So, the arc whose sine is 1/2 is 30° or, in radians, π/6. Let us see, then, the result of asin(1/2):\nasin(1/2) \\(\\displaystyle 0.523598775598299\\)\nNote that the result was not π/6. At least not symbolically: the return value was a numerical approximation, a float, of pi divided by 6. Check it on your calculator. But why? We have already talked about the problem with floats:\n1/2 \\(\\displaystyle 0.5\\)\nIn simple terms, when we pass 1/2 we are passing the float 0.5 which, internally, does not necessarily correspond to the fraction 1/2. For SymPy to understand that you really want the fraction 1/2 and not its approximate float representation, we can pass it as a string:\nasin(\u0026#39;1/2\u0026#39;) \\(\\displaystyle \\frac{\\pi}{6}\\)\nNow we got the expected result.\nAs we also saw in the first article about SymPy, in a division, if one of the operands is a SymPy object, the whole division becomes a SymPy object. That is why the following cell returns a SymPy fraction:\nacos(sympy.sqrt(3)/2) \\(\\displaystyle \\frac{\\pi}{6}\\)\nHere comes one of the great advantages of working symbolically. Remember that when there was a root in the denominator you had to go through a process called rationalizing the denominator? When we pass a SymPy object, it takes care of that internally. Note that the results of the following two cells are equal:\natan(1 / sympy.sqrt(3)) \\(\\displaystyle \\frac{\\pi}{6}\\)\natan(sympy.sqrt(3) / 3) \\(\\displaystyle \\frac{\\pi}{6}\\)\nTrigonometric identities # A considerable part of the time spent studying trigonometric functions goes to studying identities. There is a long list of trigonometric identities and I will certainly not present all of them here. The goal is just to show, with a few examples, that SymPy recognizes trigonometric identities.\nLet us start by creating symbols for the variables that will represent the angles:\nx, y = symbols(\u0026#39;x y\u0026#39;) Let us start by showing that SymPy recognizes identities involving complementary angles:\nsin(x) == cos(x - sympy.pi/2) True Now, observe what happens in the following comparison, which is one of the trigonometric identities known as the angle sum and difference identities:\nsin(x + y) == sin(x) * cos(y) + cos(x) * sin(y) False Note that the result was False. But the identity holds, so why didn\u0026rsquo;t it return True? Let us see how SymPy stores the right-hand side of the comparison above:\nsin(x) * cos(y) + cos(x) * sin(y) \\(\\displaystyle \\sin{\\left(x \\right)} \\cos{\\left(y \\right)} + \\sin{\\left(y \\right)} \\cos{\\left(x \\right)}\\)\nNote that, although we know that this sum equals sin(x + y), SymPy does not simplify it automatically. As we saw in another article, SymPy compares form, not mathematical equivalence. That is why the comparison returned False. Except for very simple cases, like the first comparison in this section of the article, it is usually recommended to compare expressions after simplifying them to get a result closer to what is mathematically expected. Let us see:\nsimplify( sin(x) * cos(y) + cos(x) * sin(y) ) \\(\\displaystyle \\sin{\\left(x + y \\right)}\\)\nsin(x + y) == simplify( sin(x) * cos(y) + cos(x) * sin(y) ) True Notice that we used the simplify function. However, there is a function specific to situations involving trigonometric functions, trigsimp:\nsin(x + y) == trigsimp( sin(x) * cos(y) + cos(x) * sin(y) ) True Even though the result was the same for both functions in this case, the documentation recommends using trigsimp when trigonometric functions are involved, since its algorithm is optimized for these situations.\nOf course, the fundamental trigonometric identity could not be missing from this article. We will use it to show once more that SymPy does not simplify automatically:\nsin(x)**2 + cos(x)**2 \\(\\displaystyle \\sin^{2}{\\left(x \\right)} + \\cos^{2}{\\left(x \\right)}\\)\nWe know that this sum equals 1, but SymPy shows the complete expression.\nLet us create an expression and assign it to a variable expr:\nexpr = 2 * sin(x)**2 + 2 * cos(x)**2 expr \\(\\displaystyle 2 \\sin^{2}{\\left(x \\right)} + 2 \\cos^{2}{\\left(x \\right)}\\)\nNotice that, by factoring out the 2, it becomes clear that this expression has value 2. But we need to explicitly ask SymPy to simplify it:\ntrigsimp(expr) \\(\\displaystyle 2\\)\nThe same holds for operations on trigonometric expressions:\nlog(expr) \\(\\displaystyle \\log{\\left(2 \\sin^{2}{\\left(x \\right)} + 2 \\cos^{2}{\\left(x \\right)} \\right)}\\)\nlog(expr).trigsimp() \\(\\displaystyle \\log{\\left(2 \\right)}\\)\ntrigsimp(log(expr)) \\(\\displaystyle \\log{\\left(2 \\right)}\\)\nNote above that we called trigsimp both as a method and directly, two equivalent forms. I personally find the second form more readable, but I am just showing that both possibilities exist.\nJust as there are simplification procedures, there are expansion procedures. Let us create an expression involving one of the multiple-angle formulas:\nexpr = sin(2*x) We saw in the introductory article about SymPy that there is the expand function:\nexpand(expr) \\(\\displaystyle \\sin{\\left(2 x \\right)}\\)\nNotice that the expression was not expanded. Here, just as with simplification, there is a specific method for expansion involving trigonometric functions, expand_trig:\nexpand_trig(expr) \\(\\displaystyle 2 \\sin{\\left(x \\right)} \\cos{\\left(x \\right)}\\)\nNow, yes, the expected result.\nTo reinforce the point, it is good practice, when trigonometric functions are involved, to use SymPy methods specific to those cases.\nPlots of sine, cosine, and tangent # I do not know if the reader is as crazy as I am, but I am hooked on plots. And the plots of trigonometric functions have always caught my attention because of their periodic nature. And one cannot truly understand the applications of these functions without a clear notion of that nature.\nLet us start by defining a standard for the plots in the article. We already saw in this article that SymPy uses Matplotlib under the hood, so by configuring Matplotlib we are indirectly also configuring the plots generated by SymPy\u0026rsquo;s plot functions:\nparams = { \u0026#39;lines.linewidth\u0026#39;: 2.0, \u0026#39;axes.labelsize\u0026#39;: 14, \u0026#39;axes.titlesize\u0026#39;: 14, \u0026#39;axes.spines.top\u0026#39;: False, \u0026#39;axes.spines.right\u0026#39;: False, \u0026#39;xtick.labelsize\u0026#39;: 14, \u0026#39;ytick.labelsize\u0026#39;: 14, \u0026#39;figure.autolayout\u0026#39;: True, \u0026#39;figure.titlesize\u0026#39;: 20, \u0026#39;figure.figsize\u0026#39;: (10, 6), \u0026#39;legend.shadow\u0026#39;: False, \u0026#39;legend.fontsize\u0026#39;: 12, } plt.rcParams.update(params) Let us first plot the sine and cosine functions:\nplot(sympy.sin(x), sympy.cos(x), # functions (x, -2*sympy.pi, 2*sympy.pi), # plot interval legend=True); # show legend with the functions OK, an expected and familiar profile. But\u0026hellip; you probably did not use x for angles in your math classes. You used Greek letters like alpha, beta, theta\u0026hellip; Since we are here anyway, let us see how to use those letters in SymPy.\nSymPy lets us pass LaTeX code to create symbols:\ntheta = Symbol(r\u0026#39;\\theta\u0026#39;) theta \\(\\displaystyle \\theta\\)\nsin(theta), cos(theta) \\(\\displaystyle \\left( \\sin{\\left(\\theta \\right)}, \\ \\cos{\\left(\\theta \\right)}\\right)\\)\nGreat! That seems to be what we want. Note that the LaTeX code was passed as a raw string (the r prefix before the string). This is so that Python does not wrongly interpret the \\ that is part of the LaTeX code.\nLet us see the plot:\nplot(sin(theta), cos(theta), (theta, -2*sympy.pi, 2*sympy.pi), legend=True); Unfortunately, in the plot generated by SymPy\u0026rsquo;s plot function the symbol does not appear, only the LaTeX code for the symbol. However, Matplotlib accepts LaTeX code; it is the passing of this SymPy symbol to Matplotlib that, for some reason, is not done correctly. But there is no problem: we already saw in this article that it is possible to extract the data from SymPy plots and pass it directly to Matplotlib. That is, eliminating SymPy as the middleman. To do this, we follow the procedure from the cited article, assigning the plot to a variable and extracting the point data and the labels:\np = plot(sin(theta), cos(theta), (theta, -2*sympy.pi, 2*sympy.pi), legend=True, show=False) plots = [] for line in p: data = line.get_points() label = line.label plots.append((data, label)) Now, let us create our own plot and pass in the data and labels we obtained:\nfig, ax = plt.subplots() for data, label in plots: ax.plot(*data, label=f\u0026#39;${label}$\u0026#39;) ax.legend(fontsize=14) ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$f(\\theta)$\u0026#39;) plt.show() Now we are talking! Note that the labels, which are LaTeX code, are surrounded by the $ symbol. This symbol tells Matplotlib that the content inside is LaTeX code.\nHowever\u0026hellip; if you remember your math classes, the plots had a scale showing the angles in radians, usually in multiples of π. Let us try, with Matplotlib\u0026rsquo;s MultipleLocator, to place the major ticks of the angle axis at multiples of π/2 and the minor ones at multiples of π/4:\nfig, ax = plt.subplots() for data, label in plots: ax.plot(*data, label=f\u0026#39;${label}$\u0026#39;) ax.legend() ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$f(\\theta)$\u0026#39;) ax.xaxis.set_major_locator(plt.MultipleLocator(np.pi / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(np.pi / 4)) plt.show() This is almost what we want. We do not want the float representation of the fractions, but for them to be shown as actual fractions. For that, we need to create a function that does this formatting and use Matplotlib\u0026rsquo;s FuncFormatter. The following function, adapted from here and from here, does the job.\ndef format_func(value, tick_number): \u0026#34;\u0026#34;\u0026#34;find number of multiples of pi/2\u0026#34;\u0026#34;\u0026#34; N = int(np.round(2 * value / np.pi)) if N == 0: return \u0026#34;0\u0026#34; elif N == 1: return r\u0026#34;$\\dfrac{1}{2}\\pi$\u0026#34; elif N == 2: return r\u0026#34;$\\pi$\u0026#34; elif N % 2 \u0026gt; 0: return r\u0026#34;$\\dfrac{{{0}}}{{2}}\\pi$\u0026#34;.format(N) else: return r\u0026#34;${0}\\pi$\u0026#34;.format(N // 2) fig, ax = plt.subplots() for data, label in plots: ax.plot(*data, label=f\u0026#39;${label}$\u0026#39;) ax.legend() ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$f(\\theta)$\u0026#39;) ax.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;) ax.xaxis.set_major_locator(plt.MultipleLocator(np.pi / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(np.pi / 4)) ax.xaxis.set_major_formatter(plt.FuncFormatter(format_func)) plt.show() Perfect! If you think it is necessary, you can add grid lines to make it easier to read:\nfig, ax = plt.subplots() for data, label in plots: ax.plot(*data, label=f\u0026#39;${label}$\u0026#39;) ax.legend() ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$f(\\theta)$\u0026#39;) ax.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;) ax.xaxis.set_major_locator(plt.MultipleLocator(np.pi / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(np.pi / 4)) ax.xaxis.set_major_formatter(plt.FuncFormatter(format_func)) ax.axhline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.axvline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.grid(which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=1.5) ax.grid(which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=1) ax.set_axisbelow(True) plt.show() Done. A plot that is easy to read and has a scale suited to the context.\nLet us now repeat the procedure for the tangent plot. This one is more challenging:\nplot(tan(theta), (theta, -2*sympy.pi, 2*sympy.pi), legend=True); Can you understand what happened here? The tangent function has a peculiarity. It does not exist when the cosine is 0 (because division by zero is not defined), so the domain is all real numbers except those that make the cosine zero.\nSo, the closer the cosine is to zero, the larger the tangent, so the function tends to infinity near those angles and is not defined at them. Basically, we are saying that there are vertical asymptotes at these angles.\nBecause of this tendency toward infinity, we see large values on the vertical axis. If we limit the plot to a smaller interval on the vertical axis, we start to see the profile we know:\nplot(tan(theta), (theta, -2*sympy.pi, 2*sympy.pi), ylim=(-4, 4), legend=True); We see that there are still lines that should not really be there. These lines are SymPy\u0026rsquo;s attempt to \u0026ldquo;connect\u0026rdquo; the positive infinite end before an undefinedness to the negative infinite end after the undefined angle. In other words, a frustrated attempt, since, in reality, the function is not continuous.\nTo solve the plotting problem, we will again need to extract the data from SymPy. Notice in the cell below that I request more points than the default with nb_of_points, since I will need to get rid of some of them (the lines):\np = plot(tan(theta), (theta, -2*sympy.pi, 2*sympy.pi), ylim=(-4, 4), legend=True, show=False, nb_of_points=1000) Let us see how Matplotlib shows the plot:\nfig, ax = plt.subplots() ax.plot(*p[0].get_points(), label=f\u0026#39;${p[0].label}$\u0026#39;, linewidth=3) ax.legend() ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$\\tan(\\theta)$\u0026#39;) ax.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;) ax.xaxis.set_major_locator(plt.MultipleLocator(np.pi / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(np.pi / 4)) ax.xaxis.set_major_formatter(plt.FuncFormatter(format_func)) ax.axhline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.axvline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.grid(which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=1.5) ax.grid(which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=1) ax.set_axisbelow(True) plt.show() As expected, the same problem. After all, we have not dealt with it yet.\nHere we will use NumPy, more specifically the masked_where method. This method lets us create what NumPy calls a mask and is ideal for handling invalid data. Basically, we will define a value beyond which values are considered invalid. In the following cell, I took that value to be 20. It is an arbitrary value, it could be larger or smaller. Since I will limit the plot between -4 and 4, I chose a slightly larger value to have more points so the curves have a smoother profile.\ntheta_values, tan_values = p[0].get_points() tan_masked = np.ma.masked_where(np.abs(tan_values) \u0026gt; 20., tan_values) fig, ax = plt.subplots() ax.plot(theta_values, tan_masked, label=f\u0026#39;${p[0].label}$\u0026#39;) ax.legend() ax.set_xlabel(r\u0026#39;$\\theta$\u0026#39;) ax.set_ylabel(r\u0026#39;$\\tan(\\theta)$\u0026#39;) ax.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;) ax.xaxis.set_major_locator(plt.MultipleLocator(np.pi / 2)) ax.xaxis.set_minor_locator(plt.MultipleLocator(np.pi / 4)) ax.xaxis.set_major_formatter(plt.FuncFormatter(format_func)) ax.axhline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.axvline(color=\u0026#34;gray\u0026#34;, zorder=-1) ax.set_ylim(-4, 4) ax.grid(which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=1.5) ax.grid(which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=1) ax.set_axisbelow(True) plt.show() Now we have the profile we know.\nWhat did we learn? A lot of Matplotlib and NumPy, I hope. And, in the case of the tangent plot, what usually happens when a function has a discontinuity. Remember: for \u0026ldquo;the computer\u0026rdquo; it is just a set of x values being passed to a function and generating pairs of points. It will do its best, or rather, what it was programmed to do, with what it was given, but it does not understand the concept of discontinuity, a human concept, a human semantics. Hence the need to process the data, removing the values beyond an arbitrary cutoff. Almost whenever \u0026ldquo;strange lines\u0026rdquo; show up, it is because of a discontinuity that went unnoticed.\nAnd another thing can be learned and is often ignored: try to reproduce material you already know in order to learn programming. You certainly already knew these plots, but did you think you would have these details to reproduce them faithfully?\n3D plots # Stop and think of a word: helix. What comes to mind? Think about the geometric shape. Can you relate it to the subject we are discussing?\nAt the beginning of the section on two-dimensional plots I highlighted the periodic profile of the plots. For this reason, the harmonic motions studied in high school physics are usually the first applied examples of trigonometric functions that we come across. At least formally, because in everyday life we have plenty of them, we just do not think about it.\nIn the Wikipedia article on the helix shape and related ones, there are some interesting figures and examples. For example, what is the first thing you learn about DNA in biology class? The famous double helix:\nDNA double helix. Source: Wikipedia Another classic example: a spring. Plants such as climbing vines with their tendrils. The path a charged particle follows in a magnetic field is also helical. Just search the internet and you will find more and more examples.\nAnd this shape can be obtained with parametric equations in Cartesian coordinates. For example:\nt = sympy.Symbol(\u0026#39;t\u0026#39;) plot3d_parametric_line(cos(t), sin(t), t, (t, 0, 6*sympy.pi), ); Simple, right? It is a periodic behavior, look, there are the sine and cosine functions.\nNow look at the following excerpts from cheesy 80s songs (in Portuguese):\nA vida vem em ondas, como um mar Num indo e vindo infinito\nSource\nTudo vem tudo vai Como as ondas do mar\nSource\nWell, besides apparently being a source of heartache for musicians, ocean waves are examples of periodic behavior, which even shows up figuratively in the lyrics. Although I prefer to remember the sea in great epic adventures, thanks to Iron Maiden.\nLook at the following plot, generated by multiplying two trigonometric functions:\nplot3d(sin(y) * cos(x), (x, -sympy.pi, sympy.pi), (y, -sympy.pi, sympy.pi),); Can you see it as a good analogy to a wave? Observe the behavior of the maxima and minima. Here chemists and physicists might recall the most classic example in those fields: atomic orbitals. You know those orbital shapes they made you memorize in high school? Look at the equations at the link: sine and cosine appear in them. Not by chance, these equations arise from the so-called wave function studied in quantum mechanics, but that is for another article.\nGo back to the previous figure. Imagine a plane parallel to the xy plane that cuts the plot. For example, take a plane at z = 0.25, with z = f(x,y) being our vertical axis. What would you see on that plane? What would the plot of this intersection, this cut, between the surface and the plane look like?\nWell, basically, we are looking for the plot of the equation sin(y) cos(x) = 0.25, right? That is an implicit function. Conveniently, SymPy has the plot_implicit method, to which we can pass the equation and get the plot:\nplot_implicit(sympy.Eq(sin(y) * cos(x), 0.25), (x, -sympy.pi, sympy.pi), (y, -sympy.pi, sympy.pi), ); Can you understand the plot? It is the set of points where the surface intersects the plane at z = 0.25, at height 0.25, at level 0.25.\nImagine that, instead of a wave, the plot represents terrain, with valleys and mountains. That is where the concept of contour lines comes from: basically, we made the contour line at height 0.25. It is an important subject in topography and a classic example in college calculus classes, usually Calculus 2.\nWhat would we see if we put several contour lines in the same plot, as if we were stacking the curves but viewing them from above, on the same plane?\nHere we will again need some help from NumPy and Matplotlib, because we want to obtain many numerical values of an equation. SymPy has the lambdify function, which lets us turn a SymPy expression into a function that can be used in NumPy numerical computations. And Matplotlib has the contour method for contour lines. Let us see:\n# create function with lambdify f = sympy.lambdify((x, y), sin(y) * cos(x), \u0026#39;numpy\u0026#39;) # create set of values x_values = np.linspace(-np.pi, np.pi, 1000) y_values = np.linspace(-np.pi, np.pi, 1000) X, Y = np.meshgrid(x_values, y_values) # create contour lines cp = plt.contour(X, Y, f(X, Y)); # put the value of each level on the plot plt.clabel(cp); The color scheme is the same one used in the 3D plot. Compare the contour lines with the 3D surface to make sure you understood what was done.\nWe can set the number of levels we want in the plot:\ncp = plt.contour(X, Y, f(X, Y), 20); plt.clabel(cp); It is very common, when we want to show many levels, to use colored maps and present a color scale. For that, we have the contourf method (the f is for filled):\nplt.contourf(X, Y, f(X, Y), 30); plt.colorbar(); A really beautiful and hallucinogenic plot. Our periodic pattern is clearly visible here, even though we have \u0026ldquo;lost\u0026rdquo; one of the dimensions of the plot.\nStill alive? Did you expect to see all this in an article about trigonometric functions? There is a bit more, but it is almost over\u0026hellip;\nHyperbolic functions # If ordinary trigonometric functions already scare some people, the hyperbolic ones even more so\u0026hellip; But, if you made it this far, it is because I managed to show how interesting, and useful, trigonometric functions are. And the hyperbolic ones are just as much so, as I intend to show.\nThe geometric origin can be seen here, but in this article I intend to go straight to the applications. So, let us briefly look at the definitions.\nHyperbolic functions can be defined from exponential functions:\n(exp(x) + exp(-x))/2 \\(\\displaystyle \\frac{e^{x}}{2} + \\frac{e^{- x}}{2}\\)\nsimplify( (exp(x) + exp(-x))/2 ) \\(\\displaystyle \\cosh{\\left(x \\right)}\\)\n(exp(x) - exp(-x))/2 \\(\\displaystyle \\frac{e^{x}}{2} - \\frac{e^{- x}}{2}\\)\nsimplify( (exp(x) - exp(-x))/2 ) \\(\\displaystyle \\sinh{\\left(x \\right)}\\)\nsimplify(sinh(x) / cosh(x)) \\(\\displaystyle \\tanh{\\left(x \\right)}\\)\nBut what is the relationship between hyperbolic and trigonometric functions? Why are they appearing in the same article? Because trigonometric functions can also be defined from exponential functions, only they are complex exponential functions according to Euler\u0026rsquo;s formula:\n$$ e^{ix} = \\cos(x) + i \\sin(x) $$Which we can represent on a complex plane:\nSymPy has the symbol I to denote the imaginary unit:\nexp(sympy.I * x) \\(\\displaystyle e^{i x}\\)\nWe can use the rewrite method to ask SymPy to rewrite a given expression using other mathematical functions:\nexp(sympy.I * x).rewrite(cos) \\(\\displaystyle i \\sin{\\left(x \\right)} + \\cos{\\left(x \\right)}\\)\nThus, we see above that Euler\u0026rsquo;s formula equality appears.\nWe can ask SymPy to rewrite the sine and cosine functions in exponential form:\nsin(x).rewrite(exp) \\(\\displaystyle - \\frac{i \\left(e^{i x} - e^{- i x}\\right)}{2}\\)\ncos(x).rewrite(exp) \\(\\displaystyle \\frac{e^{i x}}{2} + \\frac{e^{- i x}}{2}\\)\nNote that they are very similar to the definitions of hyperbolic sine and cosine, the difference being the presence of complex arguments in the exponentials.\nWell, this means that if we pass complex arguments to the sine and cosine functions, the hyperbolic sine and cosine functions will appear:\nsin(sympy.I * x).rewrite(exp) \\(\\displaystyle i \\left(\\frac{e^{x}}{2} - \\frac{e^{- x}}{2}\\right)\\)\nexpand_trig(sin(sympy.I * x)) \\(\\displaystyle i \\sinh{\\left(x \\right)}\\)\ncos(sympy.I * x).rewrite(exp) \\(\\displaystyle \\frac{e^{x}}{2} + \\frac{e^{- x}}{2}\\)\nexpand_trig(cos(sympy.I * x)) \\(\\displaystyle \\cosh{\\left(x \\right)}\\)\nWe see, therefore, that trigonometric and hyperbolic functions are closely related. I will soon write articles about complex numbers so, if you had difficulty understanding what was discussed here, I will return to the subject soon. In the meantime, this link shows the relationships between trigonometric and hyperbolic functions in more detail and may help.\nJust like trigonometric functions, there are also important identities among hyperbolic functions, for example:\ncosh(x)**2 - sinh(x)**2 \\(\\displaystyle - \\sinh^{2}{\\left(x \\right)} + \\cosh^{2}{\\left(x \\right)}\\)\nsimplify(_) \\(\\displaystyle 1\\)\nLet us see the plot of the hyperbolic sine, cosine, and tangent:\nplot(sinh(x), cosh(x), tanh(x), (x, -3, 3), ylim=(-3, 3), legend=True,); Here it starts to get (more) interesting.\nPlots of the catenary and the catenoid # Look at the following two photos:\nExample of a catenary. Source: Wikipedia Example of a catenary. Source: Wikipedia Can you relate the photos to any of the curves shown in the previous plot?\nBoth photos illustrate real examples of catenaries. A catenary is the curve assumed by a hanging chain, or flexible cable, fixed only at its ends and subject only to the force of its own weight (gravity). Catenaries are important in engineering and architecture. A force applied at any point of the curve is distributed equally throughout the material, providing greater stability to the structure. The shape is widely used in the construction of architectural arches, cathedral domes, and igloos. There are several examples at this link.\nThe mathematical definition of a catenary in Cartesian coordinates is quite simple:\na = Symbol(\u0026#39;a\u0026#39;) catenary = a * cosh(x/a) catenary \\(\\displaystyle a \\cosh{\\left(\\frac{x}{a} \\right)}\\)\nWhere a is a constant. Let us see the effect of this constant by plotting a few values:\nplots = [] for i in range(1, 6): expr = catenary.subs({a: i}) p = plot(expr, (x, -6, 6), show=False) data = p[0].get_points() plots.append(data) fig, ax = plt.subplots() for i, p in enumerate(plots, 1): ax.plot(*p, label=f\u0026#39;a = {i}\u0026#39;, linewidth=3) ax.set_ylim(0, 10) ax.legend() ax.set_xlabel(\u0026#39;x\u0026#39;) ax.set_ylabel(\u0026#39;y\u0026#39;) ax.set_title(r\u0026#34;Catenary: $y = a \\cosh \\left( \\dfrac{x}{a} \\right)$\u0026#34;) plt.show() Graphically, we see that a defines the \u0026ldquo;opening\u0026rdquo; of the curve. But, structurally, taking a chain as an example, this factor is related to the horizontal tension in the chain and to its weight. Makes sense, right?\nNow, look at the following photo:\nExample of a catenoid. Source: Wikipedia Yes, it is a soap bubble that was stretched by the blue rings in the photo. Look at the shape of the resulting surface and get ready for a mental exercise. Go back to the previous plot and pick the catenary with the widest opening, that is, with a = 5. Mentally, lay it on its side, pointing its opening to the left, and rotate it around an imaginary vertical axis. Congratulations, you just generated a catenoid in your mind, which is very similar to the phenomenon illustrated in the photo.\nA catenoid is a minimal surface, that is, a surface in which, once all the points of the boundary are fixed, any two points are joined by infinitely many curves, one of which is a catenary. Put in a way that is a bit easier to understand, it is the surface with the smallest possible area given its edges (boundary conditions). Its mean curvature is zero at every point.\nIt is a heavily studied structure, from studies of self-assembled molecular structures, through the study of tensile structures in architecture, to the evaluation of the surface tension of materials.\nIts definition in a Cartesian space can be given by the following parametric equations:\nc, v = sympy.symbols(\u0026#39;c v\u0026#39;) catenoid_x = c * cosh(v / c) * cos(t) catenoid_y = c * cosh(v / c) * sin(t) catenoid_z = v catenoid_x, catenoid_y, catenoid_z \\(\\displaystyle \\left( c \\cos{\\left(t \\right)} \\cosh{\\left(\\frac{v}{c} \\right)}, \\ c \\sin{\\left(t \\right)} \\cosh{\\left(\\frac{v}{c} \\right)}, \\ v\\right)\\)\nNote that there are trigonometric and hyperbolic functions in the definitions. We have a nonzero real constant c, a real variable t in the interval [-π, π), and a real variable v.\nLet us replace the constant c with some value, for example 1, and generate our surface:\ncatenoid_x = catenoid_x.subs({c: 1}) catenoid_y = catenoid_y.subs({c: 1}) catenoid_x, catenoid_y, catenoid_z \\(\\displaystyle \\left( \\cos{\\left(t \\right)} \\cosh{\\left(v \\right)}, \\ \\sin{\\left(t \\right)} \\cosh{\\left(v \\right)}, \\ v\\right)\\)\nplot3d_parametric_surface(catenoid_x, catenoid_y, catenoid_z, (t, -sympy.pi, sympy.pi), (v, -1, 1),); Very similar to the photo, right?\nAnd what is the effect of this constant? Let us replace it with 0.5:\ncatenoid_x = c * cosh(v / c) * cos(t) catenoid_y = c * cosh(v / c) * sin(t) catenoid_x = catenoid_x.subs({c: 0.5}) catenoid_y = catenoid_y.subs({c: 0.5}) plot3d_parametric_surface(catenoid_x, catenoid_y, catenoid_z, (t, -sympy.pi, sympy.pi), (v, -1, 1),); We see that the catenoid\u0026rsquo;s \u0026ldquo;waist got thinner.\u0026rdquo;\nConclusion and more SymPy articles # It was a long trip through trigonometric and hyperbolic functions, but I hope I made the subject interesting. Unfortunately, the teaching environment sometimes focuses too much on the mathematics and forgets to show real examples, applications, and connections between subjects. If, besides finding it interesting, you also learned a bit of Python, my goal was achieved.\nUntil next time.\n","date":"2021-12-15","externalUrl":null,"permalink":"/2021/12/trigonometric-functions-with-sympy/","section":"Blog","summary":"What do ocean waves, soap bubbles, and DNA have in common? That is what I show in this article, with the help of Python. We will see how these subjects relate to trigonometric functions.","title":"Trigonometric Functions with SymPy","type":"posts"},{"content":" Right at the start of studying any programming language, we run into the word \u0026ldquo;logic\u0026rdquo;. It is usually used to refer to the process of designing an algorithm, or to the conditional structures of the language, or to operators such as AND and OR. In these last two contexts, we are referring to what is called mathematical logic. In this article, I cover the basics of mathematical logic for programming students.\nIntroduction # This article is meant as a short introductory text on the study of mathematical logic. I tried to include what I consider the minimum starting material for those who want to go deeper later with other sources. It is built from my own notes, taken while I was studying to better understand logical operators as I learned Python. Corrections and suggestions are welcome. Those who want a more in-depth study can look for the book Filho, E. A. Iniciação à lógica matemática (in Portuguese), Editora Nobel, 2002.\nMathematical logic (also called symbolic logic or propositional logic) is the study of declarative sentences, also known as propositions. A proposition is any set of words or symbols that expresses a thought with complete meaning. For example, \u0026ldquo;The Moon is a satellite of the Earth\u0026rdquo; is a proposition, as is \u0026ldquo;π \u0026gt; √ 5 \u0026rdquo;.\nPropositions must satisfy the following two fundamental principles:\nLaw of the excluded middle: a proposition can only be true or false, with no other alternative. Law of non-contradiction: a proposition cannot be true and false at the same time. We then say that a true proposition has truth value T (true) and a false proposition has truth value F (false). Truth values are also commonly represented by 0 (zero) for false propositions and 1 (one) for true propositions.\nThe two propositions used as examples above are true, but the proposition \u0026ldquo;The number π is rational\u0026rdquo; is false.\nSome of you will already recognize aspects that appear when we start studying a programming language. Most languages have a data type called Boolean, a name derived from the mathematician George Boole, which is precisely a type that takes only two possible values.\nJust to illustrate, let\u0026rsquo;s look at some simple examples in Python:\n3 \u0026gt; 2 True 2 \u0026gt; 3 False a = 2 b = 2 a == b True Note that the comparisons above result in True or False. We can assign such values to a variable:\nt = True type(t) bool Note that the type is bool, short for Boolean.\nAn immediate application of Boolean return values is the flow control of conditional structures. Take this simple example:\npeople = {\u0026#39;João\u0026#39;: 17, \u0026#39;Maria\u0026#39;: 20, \u0026#39;Tibúrcio\u0026#39;: 18} adult_age = 18 for person, age in people.items(): if age \u0026gt;= adult_age: print(f\u0026#39;{person} can drink alcohol.\u0026#39;) else: print(f\u0026#39;{person} can NOT drink alcohol.\u0026#39;) João can NOT drink alcohol. Maria can drink alcohol. Tibúrcio can drink alcohol. Note that we created a dictionary with people\u0026rsquo;s names and ages and checked whether each person is at least 18 years old. If so, that person can drink alcohol. Note that there is a check age \u0026gt;= adult_age. The output of that check is a Boolean, which can only be True or False. The result determines the flow the program will follow, entering the if block or the else block. Let\u0026rsquo;s see the result of the check for each entry in the dictionary:\nfor person, age in people.items(): print(age \u0026gt;= adult_age) False True True Some symbols used in mathematical logic # We have already seen that the subject is related to programming. Let\u0026rsquo;s go a little deeper. Whatever language you come across, there will be the concept of logical operators, the best known being AND and OR. The symbol for each operator varies from language to language. In Python, for example, AND and OR are represented by the words and and or, respectively. In the study of logic, they are called conjunction and disjunction.\nOperators build new propositions from other ones. In logic books and materials, the following symbols are used for the main logical operators:\nSymbol Meaning ~ or \\(\\neg\\) negation \\(\\land\\) conjunction \\(\\lor\\) disjunction \\(\\rightarrow\\) conditional \\(\\leftrightarrow\\) biconditional \\( | \\) such that NOR or \\(\\downarrow\\) NOR XOR or \\(\\underline{\\vee}\\) exclusive disjunction NAND or \\(\\uparrow\\) NAND Consider the proposition \u0026ldquo;The number 6 is even and the number 8 is a perfect cube\u0026rdquo;. In bold we see a conjunction of two propositions: \u0026ldquo;The number 6 is even\u0026rdquo;, \u0026ldquo;The number 8 is a perfect cube\u0026rdquo;. It is common to represent simple propositions by lowercase letters. So, we could express the compound proposition of this example as p ^ q. But how do we determine the truth value of a given compound proposition?\nTruth tables # With the operators above, compound propositions can be created. Consequently, it is of interest to check the logical validity of a given compound proposition. That is what truth tables are for.\nThere are many online materials and books on mathematical logic for those who want a more formal presentation of the subject. In this article I want to show it in a more practical way, which the reader can easily replicate and can even use to check the results of simple logic exercises. So I will use a small Python package, ttg - truth-table-generator, which, as the name says, lets us build truth tables.\nInstallation is very simple: pip install truth-table-generator\nTo use ttg, the following terms are used as symbols. They are in single quotes to make explicit that the program receives them as strings.\nOperation Term negation 'not', '-', '~' logical disjunction 'or' logical nor 'nor' exclusive disjunction 'xor', '!=' logical conjunction 'and' logical NAND 'nand' material implication '=\u0026gt;', 'implies' logical biconditional '=' We need to import the package\u0026rsquo;s table builder, called Truths:\nfrom ttg import Truths Suppose you would like to build a truth table by hand. If a compound proposition is made of \\(n\\) simple propositions, its truth table will have \\(2^n\\) rows. In the first column, supposing it has x rows, put T (or any other mark for the true value) in the first \\(\\frac{x}{2}\\) rows and F (or any other mark for the false value) in the rest. In the second column, put T in the first \\(\\frac{x}{4}\\) rows, F in the next \\(\\frac{x}{4}\\), and keep alternating like this until the end of the rows. The same line of reasoning continues for the following columns. Practice, for example, with 4 simple propositions.\nCheck your answer against the following table generated by ttg for 4 simple propositions p, q, r and s:\nprint(Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;s\u0026#39;])) +-----+-----+-----+-----+ | p | q | r | s | |-----+-----+-----+-----| | 1 | 1 | 1 | 1 | | 1 | 1 | 1 | 0 | | 1 | 1 | 0 | 1 | | 1 | 1 | 0 | 0 | | 1 | 0 | 1 | 1 | | 1 | 0 | 1 | 0 | | 1 | 0 | 0 | 1 | | 1 | 0 | 0 | 0 | | 0 | 1 | 1 | 1 | | 0 | 1 | 1 | 0 | | 0 | 1 | 0 | 1 | | 0 | 1 | 0 | 0 | | 0 | 0 | 1 | 1 | | 0 | 0 | 1 | 0 | | 0 | 0 | 0 | 1 | | 0 | 0 | 0 | 0 | +-----+-----+-----+-----+ Notice that ttg uses 1 as the symbol for true and 0 for false by default. If we pass a parameter called ints (meaning integers: 0 and 1) as False, the table shows the words True and False instead:\nprint(Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;s\u0026#39;], ints=False)) +-------+-------+-------+-------+ | p | q | r | s | |-------+-------+-------+-------| | True | True | True | True | | True | True | True | False | | True | True | False | True | | True | True | False | False | | True | False | True | True | | True | False | True | False | | True | False | False | True | | True | False | False | False | | False | True | True | True | | False | True | True | False | | False | True | False | True | | False | True | False | False | | False | False | True | True | | False | False | True | False | | False | False | False | True | | False | False | False | False | +-------+-------+-------+-------+ This last form of representation will be the one used in the rest of the article.\nLogical operations # Let\u0026rsquo;s check the truth table for each binary operator available in ttg.\nprint(Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;], [\u0026#39;p and q\u0026#39;, \u0026#39;p or q\u0026#39;, \u0026#39;p =\u0026gt; q\u0026#39;, \u0026#39;p = q\u0026#39;, \u0026#39;p xor q\u0026#39;, \u0026#39;p nand q\u0026#39;, \u0026#39;p nor q\u0026#39;], ints=False)) +-------+-------+-----------+----------+----------+---------+-----------+------------+-----------+ | p | q | p and q | p or q | p =\u0026gt; q | p = q | p xor q | p nand q | p nor q | |-------+-------+-----------+----------+----------+---------+-----------+------------+-----------| | True | True | True | True | True | True | False | False | False | | True | False | False | True | False | False | True | True | False | | False | True | False | True | True | False | True | True | False | | False | False | False | False | True | True | False | True | True | +-------+-------+-----------+----------+----------+---------+-----------+------------+-----------+ These small tables are the ones usually shown in materials on logical operators for programming. We have already seen in this article that the short-circuit behavior of operators can be understood from these truth tables. Likewise, they are essential for understanding digital circuits and the so-called bitwise operations.\nttg can present the tables in a more visually pleasant way in Jupyter Notebooks, since it uses pandas internally. Below, two functions are declared to apply colors to the truth tables. You can change the functions if you want different colors or other effects that are not implemented. You are invited to make changes and share them.\ndef color_false_red(val): \u0026#34;\u0026#34;\u0026#34;Color red False (0) and green True (1) values\u0026#34;\u0026#34;\u0026#34; color = \u0026#39;red\u0026#39; if (val == False) else \u0026#39;green\u0026#39; return \u0026#39;color: %s\u0026#39; % color def df_style(logic, hl_rows=[], hl_cols=[]): \u0026#34;\u0026#34;\u0026#34;Applies style to logical expression (logic) pandas truth table. Text: center. Table: no index column. Highlight yellow rows and columns in lists (hl_rows and hl_cols). At the end applies color_false_red function\u0026#34;\u0026#34;\u0026#34; d = logic.as_pandas().style.set_table_styles( [{\u0026#39;selector\u0026#39;:\u0026#39;th\u0026#39;, \u0026#39;props\u0026#39;: [(\u0026#39;font-size\u0026#39;, \u0026#39;12pt\u0026#39;)]}]).set_properties( **{\u0026#39;text-align\u0026#39;: \u0026#39;center\u0026#39;, \u0026#39;font-size\u0026#39;: \u0026#39;115%\u0026#39;}).apply( lambda x: [\u0026#39;background: lightyellow\u0026#39; if x.name in hl_rows else \u0026#39;\u0026#39; for i in x], axis=1).apply(lambda x: [\u0026#39;background: lightyellow\u0026#39; if x.name in hl_cols else \u0026#39;\u0026#39; for i in x], axis=0).hide_index() d = d.applymap(color_false_red) return d Let\u0026rsquo;s see how to use these functions and take the chance to have more examples of logic. Suppose you want to build a truth table for the proposition \\((p \\land q) \\rightarrow (p \\lor q)\\). To practice, build the table by hand before continuing.\nUsing the form we have already seen, but declaring a variable so that it is easier to reproduce the table:\nproposition01 = Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;], [\u0026#39;(p and q) =\u0026gt; (p or q)\u0026#39;], ints=False) print(proposition01) +-------+-------+-------------------------+ | p | q | (p and q) =\u0026gt; (p or q) | |-------+-------+-------------------------| | True | True | True | | True | False | True | | False | True | True | | False | False | True | +-------+-------+-------------------------+ Now using the view with a pandas DataFrame table:\ndf_style(proposition01) p q (p and q) =\u003e (p or q) True True True True False True False True True False False True Suppose you would like to highlight the last column:\ndf_style(proposition01, hl_cols=[\u0026#39;(p and q) =\u0026gt; (p or q)\u0026#39;]) p q (p and q) =\u003e (p or q) True True True True False True False True True False False True Now, suppose you would like to highlight the last row:\ndf_style(proposition01, hl_rows=[4]) # the header row is counted as zero p q (p and q) =\u003e (p or q) True True True True False True False True True False False True It is possible to highlight rows and columns at the same time:\ndf_style(proposition01, hl_rows=[4], hl_cols=[\u0026#39;(p and q) =\u0026gt; (p or q)\u0026#39;]) p q (p and q) =\u003e (p or q) True True True True False True False True True False False True Worked exercise # Let\u0026rsquo;s solve an exercise where the previous ways of viewing the tables will prove useful. To practice, do the exercises by hand and compare with the result shown by ttg below.\nIf \\(p\\) is a true proposition and \\(q\\) is a false proposition, what is the truth value of the compound proposition \\((p \\land \\neg q) \\rightarrow q\\)?\nSolution: Let\u0026rsquo;s start by declaring a variable and showing a simple table with print:\nexercise01 = Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;], [\u0026#39;(p and (~q)) =\u0026gt; q\u0026#39;], ints = False) Attention: Note the care with the parentheses, especially with the negation operator.\nprint(exercise01) +-------+-------+---------------------+ | p | q | (p and (~q)) =\u0026gt; q | |-------+-------+---------------------| | True | True | True | | True | False | False | | False | True | True | | False | False | True | +-------+-------+---------------------+ The statement assigned specific truth values to the simple propositions. We can use the visual resources presented earlier to highlight the table row for those values and find the answer to the exercise:\ndf_style(exercise01, hl_rows=[2]) p q (p and (~q)) =\u003e q True True True True False False False True True False False True Therefore, the truth value of the compound proposition in the given context is False.\nTautologies, contradictions and contingencies # A tautology is any compound proposition whose truth value is always true, whatever the truth values of its simple component propositions. That is, in any context.\nA contradiction is any compound proposition whose truth value is always false, whatever the truth values of its simple component propositions. That is, in any context.\nA contingency is any compound proposition that is neither a tautology nor a contradiction.\nttg has the valuation method which, by default, analyzes the last column of a truth table and evaluates whether it is a tautology, a contradiction or a contingency. It is possible to pass parameters to valuation for tables with several columns; see the documentation for details if you are interested.\nprint(proposition01) +-------+-------+-------------------------+ | p | q | (p and q) =\u0026gt; (p or q) | |-------+-------+-------------------------| | True | True | True | | True | False | True | | False | True | True | | False | False | True | +-------+-------+-------------------------+ proposition01.valuation() 'Tautology' proposition02 = Truths([\u0026#39;p\u0026#39;], [\u0026#39;p and (~p)\u0026#39;], ints=False) print(proposition02) +-------+--------------+ | p | p and (~p) | |-------+--------------| | True | False | | False | False | +-------+--------------+ proposition02.valuation() 'Contradiction' proposition03 = Truths([\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;, \u0026#39;r\u0026#39;], [\u0026#39;(p and q) or r\u0026#39;], ints=False) print(proposition03) +-------+-------+-------+------------------+ | p | q | r | (p and q) or r | |-------+-------+-------+------------------| | True | True | True | True | | True | True | False | True | | True | False | True | True | | True | False | False | False | | False | True | True | True | | False | True | False | False | | False | False | True | True | | False | False | False | False | +-------+-------+-------+------------------+ proposition03.valuation() 'Contingency' Worked exercise # Show that the following compound propositions are tautologies.\na. \\( (p \\land q) \\rightarrow p \\)\nb. \\( p \\rightarrow (p \\lor q) \\)\nc. \\( [p \\land (p \\rightarrow q)] \\rightarrow q \\) (known as modus ponens)\nd. \\( [(p \\rightarrow q) \\land (\\neg q)] \\rightarrow (\\neg p) \\) (known as modus tollens)\nNote: These tautologies are known as rules of inference.\nexercise02 = Truths( [\u0026#39;p\u0026#39;, \u0026#39;q\u0026#39;], [\u0026#39;(p and q) =\u0026gt; p\u0026#39;, \u0026#39;p =\u0026gt; (p or q)\u0026#39;, \u0026#39;(p and (p =\u0026gt; q)) =\u0026gt; q\u0026#39;, \u0026#39;((p =\u0026gt; q) and (~q)) =\u0026gt; (~p)\u0026#39;], ints=False) print(exercise02) +-------+-------+------------------+-----------------+-------------------------+-------------------------------+ | p | q | (p and q) =\u0026gt; p | p =\u0026gt; (p or q) | (p and (p =\u0026gt; q)) =\u0026gt; q | ((p =\u0026gt; q) and (~q)) =\u0026gt; (~p) | |-------+-------+------------------+-----------------+-------------------------+-------------------------------| | True | True | True | True | True | True | | True | False | True | True | True | True | | False | True | True | True | True | True | | False | False | True | True | True | True | +-------+-------+------------------+-----------------+-------------------------+-------------------------------+ We see that they are all tautologies. We can pass each column to the valuation method and check with ttg as well:\nprint(exercise02.valuation(3)) print(exercise02.valuation(4)) print(exercise02.valuation(5)) print(exercise02.valuation(6)) Tautology Tautology Tautology Tautology Logical equivalence # Let \\(p\\), \\(q\\) and \\(r\\) be any three simple propositions. Below, the symbol \\(\\Leftrightarrow\\) is used to mean equivalence between the proposition before and the one after the symbol. Two propositions are equivalent if they have identical truth tables. So, the reader can practice by building tables for the propositions before and after the equivalence symbol in each case below and checking that they are equal.\nIdempotent laws\n\\(p \\land p \\Leftrightarrow p\\) \\(p \\lor p \\Leftrightarrow p\\) Commutative laws\n\\(p \\land q \\Leftrightarrow q \\land p\\) \\(p \\lor q \\Leftrightarrow q \\lor p\\) Identity laws\n\\(p \\land V \\Leftrightarrow p\\) \\(p \\land F \\Leftrightarrow F\\) \\(p \\lor V \\Leftrightarrow V\\) \\(p \\lor F \\Leftrightarrow p\\) Complement laws\n\\(\\neg (\\neg p) \\Leftrightarrow p\\) \\(p \\land \\neg p \\Leftrightarrow F\\) \\(p \\lor \\neg p \\Leftrightarrow V\\) \\(\\neg V \\Leftrightarrow F\\) \\(\\neg F \\Leftrightarrow V\\) Associative laws\n\\((p \\land q) \\land r \\Leftrightarrow p \\land (q \\land r)\\) \\((p \\lor q) \\lor r \\Leftrightarrow p \\lor (q \\lor r)\\) Distributive laws\n\\(p \\land (q \\lor r) \\Leftrightarrow (p \\land q) \\lor (p \\land r)\\) \\(p \\lor (q \\land r) \\Leftrightarrow (p \\lor q) \\land (p \\lor r)\\) Absorption laws\n\\(p \\land (p \\lor q) \\Leftrightarrow p\\) \\(p \\lor (p \\land q) \\Leftrightarrow p\\) De Morgan\u0026rsquo;s laws\n\\(\\neg (p \\land q) \\Leftrightarrow \\neg p \\lor \\neg q \\) \\(\\neg (p \\lor q) \\Leftrightarrow \\neg p \\land \\neg q \\) \\(\\neg (p \\land q \\land r) \\Leftrightarrow \\neg p \\lor \\neg q \\lor \\neg r \\) \\(\\neg (p \\lor q \\lor r) \\Leftrightarrow \\neg p \\land \\neg q \\land \\neg r \\) Negation of the conditional\n\\(\\neg (p \\rightarrow q) \\Leftrightarrow p \\land \\neg q\\) Negation of the biconditional\n\\(\\neg (p \\leftrightarrow q) \\Leftrightarrow (p \\land \\neg q) \\lor (\\neg p \\land q)\\) In addition, the following terms are used in the study of conditionals:\nProposition Term \\(p \\rightarrow q\\) conditional \\(q \\rightarrow p\\) converse \\(\\neg p \\rightarrow \\neg q\\) inverse \\(\\neg q \\rightarrow \\neg p\\) contrapositive The conditional and the contrapositive are equivalent: \\((p \\rightarrow q) \\Leftrightarrow (\\neg q \\rightarrow \\neg p)\\)\nLet\u0026rsquo;s check the first two De Morgan\u0026rsquo;s laws presented and the negation of the conditional with ttg.\nde_morgan01 = Truths([\u0026#39;p\u0026#39;,\u0026#39;q\u0026#39;], [\u0026#39;~(p and q)\u0026#39;, \u0026#39;(~p) or (~q)\u0026#39;]) de_morgan02 = Truths([\u0026#39;p\u0026#39;,\u0026#39;q\u0026#39;], [\u0026#39;~(p or q)\u0026#39;, \u0026#39;(~p) and (~q)\u0026#39;]) conditional_negation = Truths([\u0026#39;p\u0026#39;,\u0026#39;q\u0026#39;], [\u0026#39;~(p =\u0026gt; q)\u0026#39;, \u0026#39;p and (~q)\u0026#39;]) print(de_morgan01) +-----+-----+--------------+----------------+ | p | q | ~(p and q) | (~p) or (~q) | |-----+-----+--------------+----------------| | 1 | 1 | 0 | 0 | | 1 | 0 | 1 | 1 | | 0 | 1 | 1 | 1 | | 0 | 0 | 1 | 1 | +-----+-----+--------------+----------------+ print(de_morgan02) +-----+-----+-------------+-----------------+ | p | q | ~(p or q) | (~p) and (~q) | |-----+-----+-------------+-----------------| | 1 | 1 | 0 | 0 | | 1 | 0 | 0 | 0 | | 0 | 1 | 0 | 0 | | 0 | 0 | 1 | 1 | +-----+-----+-------------+-----------------+ print(conditional_negation) +-----+-----+-------------+--------------+ | p | q | ~(p =\u0026gt; q) | p and (~q) | |-----+-----+-------------+--------------| | 1 | 1 | 0 | 0 | | 1 | 0 | 1 | 1 | | 0 | 1 | 0 | 0 | | 0 | 0 | 0 | 0 | +-----+-----+-------------+--------------+ Notice that the two columns for each of the last three examples are equal, showing the equivalences presented.\nAdditional exercises # Some exercises are suggested below. The answer key is in a notebook in this repository (in Portuguese). In the answer key file, some remarks and discussions are also made along the solutions. I suggest that you solve the exercises by hand on paper, to practice and make sure you understood the content, and then use ttg to check your answers.\nExercise 1: Build the truth table for the propositions and classify them as tautology, contradiction or contingency.\na. \\(\\neg p \\lor \\neg q\\)\nb. \\( [ (p \\land \\neg q) \\lor \\neg r] \\land [ (\\neg p \\lor q) \\land r ] \\)\nc. \\((p \\land q) \\rightarrow (p \\lor q)\\)\nd. \\((p \\land q) \\lor r\\)\ne. \\((p \\land q) \\rightarrow p\\)\nf. \\(p \\rightarrow (p \\lor q)\\)\ng. \\( [ p \\land (p \\rightarrow q) ] \\rightarrow q \\)\nh. \\( [ (p \\rightarrow q) \\land \\neg q ] \\rightarrow \\neg q \\)\ni. \\( (p \\land q) \\land \\neg p \\)\nj. \\( [ (p \\lor \\neg q) \\land r ] \\land [ (p \\land q) \\lor \\neg r ] \\)\nk. \\( [ (p \\leftrightarrow q) \\rightarrow r ] \\leftrightarrow [ \\neg (p \\land r) \\rightarrow q ] \\)\nExercise 2: If \\(p\\) is a proposition with truth value true and \\(q\\) is a proposition with truth value false, what is the truth value of the compound proposition \\(R: (p \\land \\neg q) \\rightarrow q\\)?\nExercise 3: Assign a truth value (true or false) to each of the following statements:\na. If Mars is a planet then \\(3 = 7 - 4\\).\nb. The sum of two even numbers is an even number and \\(7^2 = 49\\).\nc. \\(3=5\\) if and only if the bear is an invertebrate animal.\nd. If \\(10^2 = 100\\) then every integer is a natural number.\ne. \\(2 = 3^2 - 7\\) or the Earth is flat.\nf. \\(3 \u0026gt; 1\\) and \\(4 \u0026gt; 2\\)\ng. \\(3 \u0026gt; 1\\) or \\(3 = 1\\)\nExercise 4: Let:\n\\(p\\): London is the capital of England. \\(q\\): The Eiffel Tower is located in London. \\(r\\): The Greenwich meridian passes through London. Translate each of the compound propositions below into natural language and determine its truth value.\na. \\(\\neg q \\land \\neg p\\)\nb. \\(\\neg q \\lor \\neg p\\)\nc. \\(\\neg (p \\land q)\\)\nd. \\(\\neg p \\lor r\\)\nExercise 5: Prove that the conditional \\(p \\rightarrow q\\) is equivalent to \\(\\neg (p \\land \\neg q)\\)\nExercise 6: Show that \\(\\neg (p \\rightarrow q)\\) is equivalent to \\(p \\land \\neg q\\)\nExercise 7: Show symbolically that the following are logically equivalent: \u0026ldquo;If a student studies, then he or she passes\u0026rdquo; and \u0026ldquo;It is not true that a student studies and does not pass\u0026rdquo;.\nExercise 8: Show symbolically that the negation of \u0026ldquo;If a student studies, then he or she passes\u0026rdquo; is \u0026ldquo;There are students who study and do not pass\u0026rdquo;.\nExercise 9: Consider the proposition: \u0026ldquo;If Edson runs for president, then he will be elected\u0026rdquo;. In which of the cases below should this conditional proposition be considered false?\na. Edson ran for president and was elected.\nb. Edson ran for president and was not elected.\nc. Edson did not run for president.\nExercise 10: Consider the conditional: \u0026ldquo;If your tooth has a cavity, you need a dentist\u0026rdquo;.\na. Suppose that \u0026ldquo;your tooth does not have a cavity and you need a dentist\u0026rdquo;. Does this mean a negation of the previous one?\nb. Write a proposition that is not a conditional and that corresponds to the negation of the proposition above.\nExercise 11: Write in symbolic language and check whether the propositions \u0026ldquo;If my name is John, then I pass the entrance exam\u0026rdquo; and \u0026ldquo;I pass the entrance exam or my name is not John\u0026rdquo; are logically equivalent.\nExercise 12: Given that the proposition \\(p \\rightarrow (r \\lor s)\\) is false and the proposition \\( (q \\land \\neg s) \\rightarrow p \\) is true, classify the statements \\(p\\), \\(q\\), \\(r\\) and \\(s\\) as true or false.\nExercise 13: Knowing that the propositions \\(p\\) and \\(q\\) are true and that the proposition \\(r\\) is false, determine the truth value of the following proposition: \\((\\neg p \\downarrow q) \\land (q \\uparrow \\neg r)\\)\nConclusion # In this article we saw the basics of mathematical logic for programmers with the help of the Python package ttg - truth-table-generator. It is a vast subject, and with the basics shown here, those who need to can research further. The notebook for this article and the one with the answers to the proposed exercises can be found in this repository (in Portuguese).\nSee you next time.\n","date":"2021-12-03","externalUrl":null,"permalink":"/2021/12/logic-for-programmers/","section":"Blog","summary":"In programming, the word “logic” can refer to the process of designing an algorithm, to the conditional structures of the language, or to logical operators such as AND and OR. In the last two cases, we are within what is called mathematical logic.","title":"Logic for Programmers","type":"posts"},{"content":" Certainly one of the most abstract subjects in math class: polynomials. After all, let whoever has never been confused by a polynomial division cast the first variable: there is x everywhere you look. In this article, we will see how the SymPy library for Python helps us handle operations involving polynomials.\nCreating polynomials manually # Let\u0026rsquo;s start by importing the library we will use throughout the article:\nimport sympy # setup for better outputs in the article, can be ignored sympy.init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.0, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;,) To begin, let\u0026rsquo;s use the most basic knowledge we acquired in the first article of the SymPy series. Let\u0026rsquo;s create a symbol for our variable and an expression that will represent a polynomial:\nx = sympy.Symbol(\u0026#39;x\u0026#39;) expr = (x - 1) * (x - 2) * (x - 3) expr \\(\\displaystyle \\left(x - 3\\right) \\left(x - 2\\right) \\left(x - 1\\right)\\)\nNote that we created our polynomial expression representing a polynomial in factored form. To obtain the expanded form, that is, the form where all the distributive multiplications are carried out, we use the expand method:\nexpr.expand() \\(\\displaystyle x^{3} - 6 x^{2} + 11 x - 6\\)\nIf we start with an expanded polynomial, we can factor it with the factor method:\n_.factor() \\(\\displaystyle \\left(x - 3\\right) \\left(x - 2\\right) \\left(x - 1\\right)\\)\nThe factored form already makes it evident that the roots of the polynomial are 1, 2, and 3. But we can obtain the roots programmatically using SymPy\u0026rsquo;s solve:\nroots = sympy.solve(expr, x) roots \\(\\displaystyle \\left[ 1, \\ 2, \\ 3\\right]\\)\nChecking equalities # Let\u0026rsquo;s create two new polynomial expressions:\nP = (x - 5) * (x + 5) Q = x**2 - 25 Note that P is simply the factored form of Q. Therefore, they are equivalent, and if we check the equality between them the result will be True, right?\nP == Q False Huh\u0026hellip;?! Well, we have already written about this equality issue in SymPy. The library checks equality of form, not mathematical equivalence. Therefore, since the forms of the two polynomials are different, the comparison returns False.\nLikewise, we know that mathematically the difference between the polynomials is zero, since they are equivalent. But the result of the following comparison is also False:\nP - Q == 0 False Why? Because SymPy is not proactive about simplifications, as we have already seen in other articles. Let\u0026rsquo;s look at the representation of the difference:\nP - Q \\(\\displaystyle - x^{2} + \\left(x - 5\\right) \\left(x + 5\\right) + 25\\)\nNote that SymPy indeed stores the expression without simplifying it. If we request the simplification with the simplify method, we see that the difference does equal zero:\nsympy.simplify(P - Q) == 0 True And we can see that one polynomial is indeed the factored version of the other with the following comparisons:\nP.simplify() == Q True P == Q.factor() True Using the polynomial constructor # So far, we have seen little that is new compared to the article about expressions with SymPy. But let\u0026rsquo;s change that. Let\u0026rsquo;s look at the type of the object Q:\nQ \\(\\displaystyle x^{2} - 25\\)\ntype(Q) sympy.core.add.Add Note that, internally, it is an object of SymPy\u0026rsquo;s Add class. That is, for SymPy such an object is simply the result of adding two other SymPy objects, which is correct. The one giving the object the meaning of a polynomial is us. Let\u0026rsquo;s unpack that last sentence.\nThe great advantage of using a high-level language such as Python is that we can write code that is closer to how we think, by using abstractions. The language interpreter takes care of translating such abstractions for the machine. So far we have used mere expressions to represent polynomials, but ideally SymPy would recognize such expressions as polynomials and not as additions. And that is possible, and we will see the advantages of this approach.\nExpressions have the as_poly method:\nQ.as_poly() \\(\\displaystyle \\operatorname{Poly}{\\left( x^{2} - 25, x, domain=\\mathbb{Z} \\right)}\\)\nLook at the result! Now SymPy recognizes it as a Poly type, recognizes that the variable is x and that the domain is the integers (it can be changed; it is just the default when we do not pass the domain explicitly). Let\u0026rsquo;s assign this result to a variable and confirm the type:\nQ_poly = Q.as_poly() type(Q_poly) sympy.polys.polytools.Poly And what is the advantage? Now we can ask for what we want with methods closer to the semantics we use in mathematics. For example, we can ask for the coefficients of the polynomial:\nQ_poly.coeffs() \\(\\displaystyle \\left[ 1, \\ -25\\right]\\)\nThe result goes from the highest degree in x to the lowest. However, notice that the result can be misleading. After all, we know there is a term of degree 1 in x; it is just that its coefficient is zero. coeffs returns only nonzero coefficients. To avoid confusion, I recommend always using the all_coeffs method:\nQ_poly.all_coeffs() \\(\\displaystyle \\left[ 1, \\ 0, \\ -25\\right]\\)\nNow we have it, all the coefficients.\nAnother concept that always shows up in the study of polynomials: the degree of the polynomial. Now that we have an abstraction closer to our language, we can simply ask for the degree with the degree method:\nQ_poly.degree() \\(\\displaystyle 2\\)\nWe can ask for the list of factors of the polynomial:\nQ_poly.factor_list() \\(\\displaystyle \\left( 1, \\ \\left[ \\left( \\operatorname{Poly}{\\left( x - 5, x, domain=\\mathbb{Z} \\right)}, \\ 1\\right), \\ \\left( \\operatorname{Poly}{\\left( x + 5, x, domain=\\mathbb{Z} \\right)}, \\ 1\\right)\\right]\\right)\\)\nAsk for the roots:\nQ_poly.all_roots() \\(\\displaystyle \\left[ -5, \\ 5\\right]\\)\nDerivatives and integrals # We can even do more elaborate operations. Moving a bit beyond high school math, it is very common to need to differentiate or integrate a polynomial. Now, such operations become trivial. Let\u0026rsquo;s look at the first derivative of our polynomial:\nQ_poly.diff() \\(\\displaystyle \\operatorname{Poly}{\\left( 2 x, x, domain=\\mathbb{Z} \\right)}\\)\nWe can obtain the value of the derivative at a given value of x using the subs method already seen in other articles:\nQ_poly.diff().subs({x: 2}) \\(\\displaystyle 4\\)\nOr, more simply, with the eval method, passing the value of x:\nQ_poly.diff().eval(2) \\(\\displaystyle 4\\)\nFor higher-order derivatives, just pass a tuple with the variable and the desired order:\nQ_poly.diff((x, 2)) \\(\\displaystyle \\operatorname{Poly}{\\left( 2, x, domain=\\mathbb{Z} \\right)}\\)\nQ_poly.diff((x, 3)) \\(\\displaystyle \\operatorname{Poly}{\\left( 0, x, domain=\\mathbb{Z} \\right)}\\)\nTo obtain the integral, we use the integrate method:\nQ_poly.integrate() \\(\\displaystyle \\operatorname{Poly}{\\left( \\frac{1}{3} x^{3} - 25 x, x, domain=\\mathbb{Q} \\right)}\\)\nLikewise, we can obtain the value of the integral at a given value of x with the subs method or the eval method:\nQ_poly.integrate().subs({x: 1}) \\(\\displaystyle - \\frac{74}{3}\\)\nQ_poly.integrate().eval(1) \\(\\displaystyle - \\frac{74}{3}\\)\nThe Poly class and complex roots # We have already seen that SymPy has the Poly class. We can pass an expression directly to the class instead of using the as_poly method:\nexpr = x**3 + 2*x + 3 A = sympy.Poly(expr) A \\(\\displaystyle \\operatorname{Poly}{\\left( x^{3} + 2 x + 3, x, domain=\\mathbb{Z} \\right)}\\)\ntype(A) sympy.polys.polytools.Poly Notice that our polynomial A has degree three. Let\u0026rsquo;s look at all of its roots:\nA.all_roots() \\(\\displaystyle \\left[ -1, \\ \\frac{1}{2} - \\frac{\\sqrt{11} i}{2}, \\ \\frac{1}{2} + \\frac{\\sqrt{11} i}{2}\\right]\\)\nWe have complex roots now. Let\u0026rsquo;s look at the result of the is_real attribute for each root:\nfor root in A.all_roots(): print(root.is_real) True False False Makes sense: the first root is real, the others are complex.\nThis explains the existence of an all_roots method. After all, if there is a method that takes care to return all the roots, there must be some method that returns only some of them, right? Depending on the application, we may be interested only in the real roots. Then we can use the real_roots method:\nA.real_roots() \\(\\displaystyle \\left[ -1\\right]\\)\nPolynomial division # Let\u0026rsquo;s check two Poly objects that we created along the article:\nA \\(\\displaystyle \\operatorname{Poly}{\\left( x^{3} + 2 x + 3, x, domain=\\mathbb{Z} \\right)}\\)\nQ_poly \\(\\displaystyle \\operatorname{Poly}{\\left( x^{2} - 25, x, domain=\\mathbb{Z} \\right)}\\)\nTo make things easier to follow, let\u0026rsquo;s assign a variable B to Q_poly:\nB = Q_poly B \\(\\displaystyle \\operatorname{Poly}{\\left( x^{2} - 25, x, domain=\\mathbb{Z} \\right)}\\)\nIn the intro of the article, we talked about polynomial division. The time has come: let\u0026rsquo;s see how to do that operation with SymPy.\nIt is quite simple, we call the div method passing the dividend and the divisor:\nsympy.div(A, B) \\(\\displaystyle \\left( \\operatorname{Poly}{\\left( x, x, domain=\\mathbb{Z} \\right)}, \\ \\operatorname{Poly}{\\left( 27 x + 3, x, domain=\\mathbb{Z} \\right)}\\right)\\)\nThe result is a tuple with two polynomials, the quotient and the remainder. We can obtain each one separately by unpacking the resulting tuple:\nquotient, remainder = sympy.div(A, B) quotient \\(\\displaystyle \\operatorname{Poly}{\\left( x, x, domain=\\mathbb{Z} \\right)}\\)\nremainder \\(\\displaystyle \\operatorname{Poly}{\\left( 27 x + 3, x, domain=\\mathbb{Z} \\right)}\\)\nSince this is a division, we expect that divisor × quotient + remainder = dividend. Let\u0026rsquo;s see:\nB * quotient + remainder \\(\\displaystyle \\operatorname{Poly}{\\left( x^{3} + 2 x + 3, x, domain=\\mathbb{Z} \\right)}\\)\nA == B * quotient + remainder True Simple.\nConclusion and more articles about SymPy # In this article we saw the basics of how to handle polynomials in SymPy, and I hope it was possible to see the power of this library.\nSee you next time.\n","date":"2021-11-22","externalUrl":null,"permalink":"/2021/11/polynomials-sympy-python-calculations/","section":"Blog","summary":"In this article, we will see how the SymPy library for Python helps us handle operations involving polynomials.","title":"Polynomials with SymPy - Calculations with Python","type":"posts"},{"content":"","date":"2021-11-15","externalUrl":null,"permalink":"/categories/brazilian-exam-problems/","section":"Categories","summary":"","title":"Brazilian Exam Problems","type":"categories"},{"content":"","date":"2021-11-15","externalUrl":null,"permalink":"/categories/enem-tecnologico/","section":"Categorias","summary":"","title":"ENEM Tecnológico","type":"categories"},{"content":" Is it possible to study for the college entrance exam and still learn Python? Of course!\nIn this article, we analyze a problem from the 2020 exam. It is a chemistry problem, but one that can be solved more with reasoning than with chemistry itself. The exam is the ENEM (Exame Nacional do Ensino Médio), the national high school exam that Brazilian students take to enter universities.\nThe problem statement # Below is the problem statement, taken from the regular yellow exam booklet given in 2020 (the original is in Portuguese):\nSucrase (or invertase) is an enzyme that acts in the human intestine, hydrolyzing the disaccharide sucrose into the monosaccharides glucose and fructose. In a kinetic study of the hydrolysis reaction of sucrose (C12H22O11), 171 g of sucrose were dissolved in 500 mL of water. It was observed that, every 100 minutes of reaction, the sucrose concentration was reduced by half, regardless of the moment chosen as the initial time. The molar masses of the elements H, C, and O are equal to 1, 12, and 16 g.mol−1, respectively. What is the sucrose concentration 400 minutes after the start of the hydrolysis reaction?\nThe options are:\nA. 2.50 × 10−3 mol.L−1 B. 6.25 × 10−2 mol.L−1 C. 1.25 × 10−1 mol.L−1 D. 2.50 × 10−1 mol.L−1 E. 4.27 × 10−1 mol.L−1 Let\u0026rsquo;s recognize a few things in the problem. First, the ENEM pattern of having a story with information that is not needed to solve the problem. In this article, I will assume that the reader has a high school level of education, whether as a student, or someone interested in science but without advanced training in the area, or even a high school teacher. So the beginning of the problem may seem a bit challenging. After all, what the heck is hydrolysis, disaccharide, monosaccharide\u0026hellip;? The student is not necessarily used to these words.\nThree steps to understand problems # The first step, then, is to recognize fundamental concepts that will let us focus on the core of the problem itself. Thus, we recognize that this is a kinetics problem (\u0026quot;\u0026hellip; kinetic study\u0026hellip;\u0026quot;). At this point, some keywords should come to mind:\nreaction order (first, second, zero\u0026hellip;); rate law; catalyst; half-life. A good part of the high school level kinetics problems focus on these keywords so, if you are a student and are going to take the ENEM, in kinetics this is what you need to know.\nSo, the second step is to check which keyword connects with the problem. Note the passage \u0026ldquo;\u0026hellip;every 100 minutes of reaction, the sucrose concentration was reduced by half\u0026hellip;\u0026rdquo;. Well, what is basically being said is that the half-life of the reaction is 100 minutes. Using the Wikipedia article on half-life as a reference, we have the following explanation for the term: if the quantity that decays has a value at the start of the process, at the half-life the quantity will have half of that value.\nWe already know that this is a kinetics problem, more specifically about half-life. The last step before moving on to the solution is to observe the system of units of the problem and what was provided. Neglecting this step is a significant source of errors. Note that the question is about concentration and the options are in mol.L-1. However, the initial concentration was not given, but rather the initial mass. So, there will be a calculation to find the initial concentration, which will involve the volume. The volume provided is in mL but the options are in liters. So, there will be a unit conversion.\nThat\u0026rsquo;s it, we now know everything we need and all that is left is to do the calculations. Note that, after recognizing the context and the topic of the problem, everything else plays a merely decorative role. Rarely, in the ENEM, does the student actually need to understand the scenario that was presented and all the \u0026ldquo;fluff\u0026rdquo;. Once the keywords are captured, the problem usually becomes substituting values into simple equations or carrying out simple reasoning.\nLet\u0026rsquo;s go to Python!\nSolving the problem # Let\u0026rsquo;s start by importing the libraries we will use. The role of each one will be explained throughout the article:\nimport numpy as np import matplotlib.pyplot as plt import matplotlib.ticker as plticker from molmass import Formula Note that, at the end of the statement, the molar masses of the elements that make up sucrose were provided. The goal is to let the student calculate the molar mass of sucrose, needed to calculate the initial concentration of the solution. This calculation is very simple: just multiply each element\u0026rsquo;s molar mass by the number of times it appears in the formula and add up the values obtained. But\u0026hellip;\nThink. You have a computer that is dozens of orders of magnitude more powerful than your pocket calculator. Millions of crazy chemists around the world do molar mass calculations all the time. Surely someone has already gone to the trouble of making their routine easier by creating a package that speeds up this calculation. For that, there is molmass. See how simple it is: we pass a string with the compound\u0026rsquo;s formula to the Formula class of molmass and request the mass attribute:\nformula = Formula(\u0026#39;C12H22O11\u0026#39;) formula.mass # g/mol 342.297037 An advantage is that the package uses the molar masses of the elements without any rounding, avoiding the propagation of errors from intermediate roundings.\nNote that I put the unit in a comment. It is good practice to make the system of units of the code explicit. I have already written on this site about the pint package that lets you work with units in the code itself, converting them automatically. I could use it in this article too, but I will do the conversions step by step to make the article more didactic, closer to what the student would do on paper.\nNow, let\u0026rsquo;s create variables for each piece of information in the statement. Once again, notice the care in including the units. It is a simple detail that helps other people understand your code and helps you notice problems of unit inconsistency (dimensional analysis). And it also helps another very important person: your own future \u0026ldquo;self\u0026rdquo;, who won\u0026rsquo;t even remember what you were thinking when you wrote the code ;-)\nmass = 171 # g volume = 500 # mL half_life = 100 # min time = 400 # min We have already seen that the volume must be in liters. So, we can redefine the variable volume so that it is its current value times 10-3. After all, the prefix m in mL means exactly 10-3 x L:\nvolume = volume * 1E-3 # mL to L The 1E-3 in the code has the same meaning as 1 x 10-3. This E notation is widely used in calculators and computer programs for powers of 10.\nNow we have enough to calculate the initial molar concentration of sucrose. From chemistry classes:\n$$\\begin{aligned} \u0026\\text{molar concentration} \\\\ \u0026\\quad = \\frac{\\text{mass}}{\\text{molar mass} \\cdot \\text{volume}} \\\\ \u0026\\quad = \\frac{g}{\\frac{g}{mol} \\cdot \\ell} = \\frac{mol}{\\ell} \\end{aligned}$$Note that, if you don\u0026rsquo;t remember the formula, you can figure out from the units alone how to arrange the quantities to give the desired unit. In this case, mol/ℓ. So, let\u0026rsquo;s do this calculation:\ninitial_concentration = mass / (formula.mass * volume) initial_concentration # mol/l 0.9991322244486738 Now, let\u0026rsquo;s deal with the half-life and finish solving the problem. The concept of half-life is simple and we can reach the answer with reasoning alone. We have already seen that, after each half-life period, the concentration drops by half. Thus:\n$$\\begin{aligned} c \u0026\\xrightarrow{100\\ \\text{min}} \\frac{c}{2} \\\\ \\frac{c}{2} \u0026\\xrightarrow[\\text{total: } 200\\ \\text{min}]{100\\ \\text{min}} \\frac{\\frac{c}{2}}{2} = \\frac{c}{2^2} = \\frac{c}{4} \\\\ \\frac{c}{4} \u0026\\xrightarrow[\\text{total: } 300\\ \\text{min}]{100\\ \\text{min}} \\frac{\\frac{c}{4}}{2} = \\frac{c}{2^3} = \\frac{c}{8} \\\\ \\frac{c}{8} \u0026\\xrightarrow[\\text{total: } 400\\ \\text{min}]{100\\ \\text{min}} \\frac{\\frac{c}{8}}{2} = \\frac{c}{2^4} = \\frac{c}{16} \\end{aligned}$$$$c \\equiv \\text{initial concentration}$$So, after 400 min, the initial concentration was divided by 16:\ninitial_concentration / 16 # mol/l 0.06244576402804211 We have reached our answer! Option B, 6.25 x 10-2 mol L-1. But let\u0026rsquo;s make the code a bit better and more general.\nGeneralizing the solution # Note, in the diagram above, that there is a relationship involving powers of 2 and the number of half-life periods that have elapsed. So, we can generalize our solution a bit more. The number of half-life periods can be easily calculated:\nnum_half_lives = time / half_life num_half_lives 4.0 And, with that, calculate the final concentration:\nfinal_concentration = initial_concentration / 2**num_half_lives final_concentration # mol/l 0.06244576402804211 Same result as before, but note that the \u0026ldquo;magic number\u0026rdquo; 16 no longer appears. We managed to express the problem based on its fundamentals. And, of course, we can present the result in a friendlier way, without so many numbers that are not actually significant, and with the unit. For that, a bit of string formatting:\nprint(f\u0026#39;{final_concentration:.2E} mol/L\u0026#39;) 6.24E-02 mol/L Much better.\nHowever\u0026hellip; think! It would be very interesting if we could somehow just provide the input data and have the result appear. That is what computers are for: to waste hours watching videos on YouTube and to automate repetitive tasks.\nWe already know the fundamentals of the problem, so we can write a function that takes the input data (initial concentration, half-life, and time) and returns the final concentration:\ndef concentration_after_half_lives(initial_concentration, half_life, time): num_half_lives = time / half_life return initial_concentration / 2**num_half_lives concentration_after_half_lives(initial_concentration, half_life, time) 0.06244576402804211 Now we have a function that automates all the work we have done. It can be used in any similar exercise. It can even be put in a Python script and reused in other contexts that are, mathematically, similar, as we will see next.\nSince we are here, let\u0026rsquo;s understand a bit more about kinetics.\nA bit more about kinetics # Several simplifications are made in the study of chemistry at the high school level. One of them is treating the concept of half-life as tied to first-order kinetics reactions.\nConceptually, any order can have an expression for half-life. However, the concept is more important from a practical point of view in first-order kinetics such as, for example, radioactive decay or tracking the concentration of some drugs in the bloodstream. Let\u0026rsquo;s, then, recall important points of first-order kinetics and relate them to the problem.\nConsidering any chemical species A, if it undergoes any transformation that obeys first-order kinetics, then the following equations are valid:\n$$v = k[A]$$$$[A]_t = [A]_0 e^{-kt}$$$$t_{\\frac{1}{2}} = \\frac{\\ln 2}{k}$$$$\\begin{aligned} v \u0026\\equiv \\text{rate} \\\\ k \u0026\\equiv \\text{rate constant} \\\\ [A]_t \u0026\\equiv \\text{concentration at time } t \\\\ [A]_0 \u0026\\equiv \\text{concentration at the initial time} \\\\ t_{\\frac{1}{2}} \u0026\\equiv \\text{half-life} \\end{aligned}$$Can you recognize the powers of 2 seen earlier in any of these equations?\nSo, we can write functions that give the rate constant for the reaction and the concentration of A (in the problem, sucrose) after any time. In the functions, I used the NumPy library for the ln (log) and exponential (exp) functions:\ndef first_order_rate_constant(half_life): return np.log(2) / half_life def concentration(initial_concentration, half_life, time): rate_constant = first_order_rate_constant(half_life) return initial_concentration * np.exp(-rate_constant * time) first_order_rate_constant(half_life) # unit will be min^-1 0.006931471805599453 concentration(initial_concentration, half_life, time) # mol/l 0.06244576402804211 Obviously, the result obtained is the same as before. But now we have used a deeper knowledge of the subject. Note that it is perfectly possible to solve the problem without going into reaction order (as we did before). However, knowing that this is a first-order kinetics problem, we can explore a bit more. Let\u0026rsquo;s visualize graphically what \u0026ldquo;halving the concentration\u0026rdquo; means and, with the visualization, understand what exponential decay means, a term widely used in first-order kinetics.\nFirst, let\u0026rsquo;s obtain the sucrose concentrations after 5 consecutive half-lives to have some reference points on the plot:\ntimes = np.array([time for time in range(0, 501, 100)]) concentrations = concentration(initial_concentration, half_life, times) concentrations array([0.99913222, 0.49956611, 0.24978306, 0.12489153, 0.06244576, 0.03122288]) Can you find a pattern in the values above?\nNow, let\u0026rsquo;s use the Matplotlib library to build a plot. In the plot, I want the points above to stand out and I want a curve that shows the profile of an exponential function. Here is the code:\n# configuring Matplotlib so the plot looks more presentable params = { \u0026#39;lines.linewidth\u0026#39;: 2.5, \u0026#39;axes.labelsize\u0026#39;: 16, \u0026#39;axes.titlesize\u0026#39;: 16, \u0026#39;xtick.labelsize\u0026#39;: 14, \u0026#39;ytick.labelsize\u0026#39;: 14, \u0026#39;figure.autolayout\u0026#39;: True, \u0026#39;figure.titlesize\u0026#39;: 24, \u0026#39;figure.figsize\u0026#39;: (12, 8), \u0026#39;legend.shadow\u0026#39;: False, \u0026#39;legend.fontsize\u0026#39;: 14, } plt.rcParams.update(params) fig, ax = plt.subplots() # creating points for a theoretical curve t = np.array([time for time in range(0, 551, 20)]) c = concentration(initial_concentration, half_life, t) ax.plot(t, c, zorder=1, label=\u0026#39;Theoretical curve - 1st-order kinetics\u0026#39;) # the half-life points created earlier ax.scatter(times, concentrations, color=\u0026#39;red\u0026#39;, s=60, zorder=2, label=\u0026#39;Half-life points\u0026#39;) # creating a grid with suitable spacing to make the plot easier to read loc = lambda x: plticker.MultipleLocator(base=x) ax.xaxis.set_major_locator(loc(50)) ax.yaxis.set_major_locator(loc(0.25)) ax.yaxis.set_minor_locator(loc(0.125)) ax.grid(b=True, axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=1.5) ax.grid(b=True, which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=1.0) ax.set_axisbelow(True) # properly labeling the plot and its components ax.set_xlabel(\u0026#39;Time / min\u0026#39;) ax.set_ylabel(\u0026#39;Concentration / $mol \\cdot L^{-1}$\u0026#39;) fig.suptitle(\u0026#39;Sucrose inversion\u0026#39;) plt.legend() plt.show() Follow each red point on the resulting plot. Note that, every 100 min (our half-life in the problem), the concentration drops to half of the value it had before those 100 min had passed. Use the grid lines to make it easier to read. Any 100 min interval could be used; for convenience, we started from the value at 0 min.\nAs said earlier, the most common example of first-order kinetics is radioactive decay; we see in the plot the reason why this decay is usually called exponential.\nSo, what is sucrose inversion? # To finish, let\u0026rsquo;s understand what this hydrolysis is and why the enzyme can be called invertase. Yes, it is an enzyme because the ending is \u0026ldquo;ase\u0026rdquo; and enzymes are catalysts. Remember the keywords I showed at the beginning as being important in kinetics? There it is, catalysts\u0026hellip;\nWhenever you study a reaction, write its equation with the structures. Always. No exceptions.\nWhat would you do, then? Well, you would probably google the names of the substances, look at the structures, and copy them to build the equation. Or you would spend your hard-earned money buying programs to draw chemical structures. But time is pressing and laziness reigns. Life would be easier if we could enter the names of the substances and the computer would build the equation. And that is what we will do.\nThe article would get too long and dense if I detailed how chemistry packages work internally to give the computer all the information present in a chemical structure in order to draw it. If organic chemistry classes have already traumatized you with the naming rules, imagine encoding those names. I will explain briefly, but if you want an article just about that, leave it in the comments of the article or on social media.\nIn simplified terms, there is a naming specification called SMILES that incorporates in a string all the information needed to reproduce a given chemical structure. And, fortunately, there are huge online databases, such as PubChem, from which such SMILES can be extracted for millions of chemical compounds. The pubchempy package lets us search PubChem programmatically.\nSo, let\u0026rsquo;s search for the names of the substances and store each SMILES in a list. The statement says that sucrose is transformed into fructose and glucose. But, of course, the names must be in English for the search to work. Here is the code:\nimport pubchempy names = (\u0026#39;sucrose\u0026#39;, \u0026#39;water\u0026#39;, \u0026#39;fructose\u0026#39;, \u0026#39;glucose\u0026#39;) smiles = [] for name in names: smiles.append(pubchempy.get_compounds(name,\u0026#39;name\u0026#39;)[0].isomeric_smiles) smiles ['C([C@@H]1[C@H]([C@@H]([C@H]([C@H](O1)O[C@]2([C@H]([C@@H]([C@H](O2)CO)O)O)CO)O)O)O)O', 'O', 'C1[C@H]([C@H]([C@@H](C(O1)(CO)O)O)O)O', 'C([C@@H]1[C@H]([C@@H]([C@H](C(O1)O)O)O)O)O'] Believe it or not, this tangle of symbols is all the information needed to (re)build each structure.\nNow, we need a package that can understand these SMILES and turn them into pictures of the structures in the form of an equation. This is where RDKit comes in:\nfrom rdkit.Chem import AllChem from rdkit.Chem.Draw import IPythonConsole IPythonConsole.ipython_useSVG=True IPythonConsole.molSize = 550,200 reaction = AllChem.ReactionFromSmarts(f\u0026#39;{smiles[0]}.{smiles[1]}\u0026gt;\u0026gt;{smiles[2]}.{smiles[3]}\u0026#39;, useSmiles=True) reaction Here is our reaction equation, practically effortlessly.\nTranslating the statement (with some simplifications of language): it is a hydrolysis because a chemical bond is broken in the presence of water (from the Greek: hydro: water, lysis: breaking). Sucrose is a sugar in everyday language. The more formal name of the class of compounds that sugars belong to is carbohydrates, compounds made of carbon, hydrogen, and oxygen, also called saccharides. Look at the structure of sucrose and see that there are two rings joined by bonds involving an oxygen atom. Each of the linked groups is a monosaccharide. Therefore, sucrose is a disaccharide (two linked monosaccharides). I hope this helped to decipher the statement. Consult your nearest chemistry teacher for more details and more rigorous language.\nWhy is the enzyme called invertase? If you have studied organic chemistry, you should remember that there is the concept of chirality and should recognize that in the structures shown there are several chiral centers on carbon atoms. When a beam of polarized light, a concept seen in physics, passes through a solution containing chiral species, the polarization angle can change. See the following figure:\nWorking principle of a polarimeter. 1. Light source; 2. Unpolarized light; 3. Polarizer; 4. Polarized light; 5. Sample solution; 6. Rotation of light due to chiral molecules; 7. Rotating analyzer; 8. Detector. Source: Wikipedia In the case of the problem, when there is a solution with only sucrose, the polarized light is deflected in one direction. However, as sucrose turns into the mixture of fructose and glucose, the deflection angle decreases because the direction of the deflection changes. Hence the idea of inversion, which refers to the change in the direction of the polarized light. Again, I simplified the language quite a bit to make the text more accessible; consult your nearest physics or chemistry teacher for more details and more rigorous language.\nConclusion # Impressive how much information can be extracted from a statement, right? The problem itself is simple but, each time you question something, new knowledge is gained.\nSo, did you learn more Python or more chemistry? For more ENEM problems solved with Python click here.\nShare this article with students and with anyone else you like.\nSee you next time!\n","date":"2021-11-15","externalUrl":null,"permalink":"/2021/11/enem-python-sucrose-inversion-2020/","section":"Blog","summary":"Is it possible to study for the college entrance exam and still learn Python? Of course! In this article, we analyze a problem from the 2020 exam. It is a chemistry problem, but one that can be solved more with reasoning than with chemistry itself.","title":"ENEM with Python: The Sucrose Inversion Problem (2020)","type":"posts"},{"content":"","date":"2021-11-15","externalUrl":null,"permalink":"/tags/rdkit/","section":"tags","summary":"","title":"RDKit","type":"tags"},{"content":" In this article we will see how to make plots with SymPy and Matplotlib.\nWhen people talk about SymPy, we usually focus on solving mathematical problems. But many forget that the library has some basic features for building plots, which we will see in this article.\nCreating an equation # To illustrate, I will use the ideal gas law in this article. The famous PV=nRT from high school. Let\u0026rsquo;s see how to make plots that show how the volume of an ideal gas depends on pressure and on temperature.\nLet\u0026rsquo;s start by importing the libraries we will use throughout the article. The role of each one will be explained along the way:\nimport sympy from sympy.plotting import plot, plot3d import numpy as np import matplotlib.pyplot as plt from matplotlib import animation # setup for better outputs in the article, can be ignored sympy.init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.0, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;,) Just for reference, the SymPy version in this article is 1.9, the latest at the time of writing.\nsympy.__version__ '1.9' In case you don\u0026rsquo;t remember, the gas constant R has several possible values depending on the system of units used. You have most likely used the value \u0026ldquo;0.082\u0026rdquo; many times in your life, since it is the value associated with the units atmosphere (for pressure) and liters (for volume), which are the most common in Brazil for these quantities. The temperature is expressed in Kelvin. In fact, this is an approximate value that makes calculations easier by hand or on simple calculators. Here, we don\u0026rsquo;t need to approximate, so let\u0026rsquo;s put in all the significant digits:\ngas_constant = 0.082057366080960 # atm L / (mol K) In fact, we should not approximate. Never make approximations in intermediate steps, it will only lead to propagated errors in your calculations. But that is a subject for another article. Just take advantage of having a computer that handles numbers very well and don\u0026rsquo;t approximate. If you need a \u0026ldquo;well-behaved\u0026rdquo; number for an answer, round only at the end.\nBack to the point, let\u0026rsquo;s now create symbols for each quantity and create our equation. If you have questions about this step, see the article about equations with SymPy:\nP, V, T, R, n = sympy.symbols(\u0026#39;P V T R n\u0026#39;, positive=True) ideal_gas_law = sympy.Eq(P*V, n*R*T) ideal_gas_law \\(\\displaystyle P V = R T n\\)\nMaking the role of each symbol clear:\nP: pressure in atmospheres (atm) V: volume in liters n: amount of substance in mol R: gas constant in atm.l/(mol.K) T: temperature in Kelvin Let\u0026rsquo;s solve this equation for the volume V:\nsympy.solve(ideal_gas_law, V) \\(\\displaystyle \\left[ \\frac{R T n}{P}\\right]\\)\nNote that the result comes out as a list with a single element, which is the expression resulting from isolating V on the left-hand side of the equality. Let\u0026rsquo;s assign this expression to a variable:\nvolume = sympy.solve(ideal_gas_law, V)[0] # position 0 (first) of the list volume \\(\\displaystyle \\frac{R T n}{P}\\)\nThere are three variables in the expression. Put another way, V=f(P, n,T), volume is a function of pressure, amount of substance, and temperature. Let\u0026rsquo;s start simple, obtaining an expression for constant temperature and amount of substance. To do that, we substitute 1 mol and 25 °C (298.15 K) into the expression:\nmolar_volume_25_celsius = volume.subs({R: gas_constant, n: 1, T: 298.15}) molar_volume_25_celsius \\(\\displaystyle \\frac{24.4654036970382}{P}\\)\nNow we are able to make a plot showing how the volume V changes with variations in the pressure P, keeping n and T constant at 1 mol and 298.15 K, respectively.\nPlots with SymPy # At the beginning of the article, we saw that we imported SymPy\u0026rsquo;s plot method. Let\u0026rsquo;s pass the expression we obtained for the volume as an argument and indicate which variable to use, in this case P:\nplot(molar_volume_25_celsius, P) There are some interesting observations about the generated plot:\nThose who are more used to Python must have recognized the \u0026ldquo;look\u0026rdquo; of Matplotlib in the plot. And, indeed, SymPy uses Matplotlib as the backend for plots Notice how the axes were named. The abscissa (the \u0026ldquo;x axis\u0026rdquo;) was named P, the variable indicated when calling the plot method. The ordinate (\u0026ldquo;y axis\u0026rdquo;) was named f(P), that is, indicating that this axis is a function of P. Since it is a mathematical library, SymPy assumes that y = f(x) and makes this explicit in the plot. It is possible to change these labels, but it is still important to note the default representation. Another characteristic of a mathematical library, the plot is centered at the origin (0, 0). This is not the Matplotlib default, it is a modification made by SymPy. But, perhaps, what draws the most attention is the fact that there is a part of the plot at negative values. Obviously, physically this makes no sense since we are dealing with pressure and volume. In fact, when creating the symbols at the beginning we set positive = True. So, why did a plot appear in the 3rd quadrant? In the documentation of the plot method we see that, by default, the range of domain values considered is (-10, 10), as can be seen in the plot. Therefore, regardless of what we asked SymPy to assume when creating the symbols, the plot will be generated for the range requested by Matplotlib. But that is a practical explanation.\nIn fact, we must keep in mind that all these assumptions we ask SymPy to make, for example, that a given symbol has only positive values, are only valid within the context of SymPy operations. SymPy is a symbolic abstraction layer over plain Python. When it interacts with other libraries, as in the case of Matplotlib, there is no way to guarantee behaviors consistent with the assumptions. The assumptions are valid within SymPy operations and are important as we saw in the article about logarithms. However, be careful with your expectations when interacting with other libraries.\nWith these caveats made, let\u0026rsquo;s restrict the domain to a range that makes more sense. To do that, we just pass our variable and the minimum and maximum values in a tuple:\nplot(molar_volume_25_celsius, (P, 0.1, 4)) We already have the characteristic isotherm profile of an ideal gas. If we want to restrict the plot even further, to a range of smaller volume values, we just pass the desired minimum and maximum values for the y axis to the ylim argument in the form of a tuple:\nplot(molar_volume_25_celsius, (P, 0.1, 4), ylim=(0, 80)) And what if we want another curve in the plot? For example, for 100 °C (373.15 K)? Simple, just create the expression and pass it to the plot method:\nmolar_volume_100_celsius = volume.subs({R: gas_constant, n: 1, T: 373.15}) plot(molar_volume_25_celsius, molar_volume_100_celsius, (P, 0.1, 4), ylim=(0, 80)) We see that a second curve was created, for the new temperature. We could keep adding curves following the procedure above and, of course, improve the presentation of the plot by identifying the curves. However, it will soon become evident that interacting with Matplotlib through SymPy is quite bureaucratic and limited. For example, it makes little sense to create one expression per curve if the goal is, say, to show curves for 5 temperatures in the plot. And this is not a criticism of SymPy, handling plots is simply not the focus of the library, they already went a long way in providing a simplified way to generate plots. The point is that more advanced customizations become much simpler if we extract the data from the SymPy objects and pass it directly to Matplotlib.\nPlots with Matplotlib # Let\u0026rsquo;s unlock the unlimited power of customization of Matplotlib.\nThanks, Palpatine, we\u0026rsquo;re going to the Matplotlib side of the force\u0026hellip; Let\u0026rsquo;s see how this works.\nThe first step is to ask the plot method not to display the plot itself:\nplot(molar_volume_25_celsius, (P, 0.1, 4), ylim=(0, 80), show=False) \u0026lt;sympy.plotting.plot.Plot at 0x7f46a254fa00\u0026gt; Note that only a Plot object at a memory address is generated. So, we can assign this object to a variable. Let\u0026rsquo;s create, then, a variable with the super creative name of p:\np = plot(molar_volume_25_celsius, (P, 0.1, 4), ylim=(0, 80), show=False) p \u0026lt;sympy.plotting.plot.Plot at 0x7f46a26c3280\u0026gt; We can order the plot to be displayed from this variable:\np.show() Now, to understand the next steps, let\u0026rsquo;s understand in a very simplified way the composition of a Matplotlib plot. When you look at a plot like the one above, think of it as:\na blank rectangle, which is the base of the figure a system of axes on top of this rectangle the system of axes has the line element, the labels of each axis, the values shown on the axis, and everything else we see if desired, new axes systems can be added to the same base figure and, if desired, new lines (or points) can be added to a given axis This is basically how plots are created with Matplotlib. More details can be seen in the Matplotlib API documentation and in more rigorous language. The goal here is to understand that we can decompose a plot into more fundamental elements (objects in programming).\nThus, the object associated with our variable p is, in fact, a container! It has all these requirements to create a Matplotlib plot. Let\u0026rsquo;s see all the public attributes and methods associated with this object:\n[x for x in dir(p) if not x.startswith(\u0026#39;_\u0026#39;)] ['annotations', 'append', 'aspect_ratio', 'autoscale', 'axis', 'axis_center', 'backend', 'extend', 'fill', 'legend', 'margin', 'markers', 'rectangles', 'save', 'show', 'size', 'title', 'xlabel', 'xlim', 'xscale', 'ylabel', 'ylim', 'yscale', 'zlabel'] Well, it\u0026rsquo;s all there. The information about the axes, the labels, (almost) everything we talked about earlier. For example, let\u0026rsquo;s check the y axis label and the limit we requested (from 0 to 80):\np.ylabel, p.ylim \\(\\displaystyle \\left( f{\\left(P \\right)}, \\ \\left( 0.0, \\ 80.0\\right)\\right)\\)\nGreat! Now, there are two \u0026ldquo;suspicious\u0026rdquo; methods in this list: append and extend. They are two methods characteristic of lists, associated with the ability to grow a list by adding more elements. So\u0026hellip; the object p was created with a list structure. And what will the content of this list be? Let\u0026rsquo;s look at the first position of the list:\np[0] \u0026lt;sympy.plotting.plot.LineOver1DRangeSeries at 0x7f46a2620a00\u0026gt; Bingo! A LineOver1DRangeSeries object, that is, spelled out, a line over a range of one-dimensional values. Translating, our curve. And, since there is only one curve, it makes no sense to have an element at the next position, right?\np[1] --------------------------------------------------------------------------- IndexError Traceback (most recent call last) /tmp/ipykernel_8614/3900177224.py in \u0026lt;module\u0026gt; ----\u0026gt; 1 p[1] /opt/anaconda/lib/python3.8/site-packages/sympy/plotting/plot.py in __getitem__(self, index) 249 250 def __getitem__(self, index): --\u0026gt; 251 return self._series[index] 252 IndexError: list index out of range Right! So, we already know that new curves are new elements in this list that is the object p. And what public attributes and methods are available for each element of this list?\n[x for x in dir(p[0]) if not x.startswith(\u0026#39;_\u0026#39;)] ['adaptive', 'depth', 'end', 'expr', 'get_color_array', 'get_data', 'get_points', 'get_segments', 'is_2Dline', 'is_3D', 'is_3Dline', 'is_3Dsurface', 'is_contour', 'is_implicit', 'is_line', 'is_parametric', 'label', 'line_color', 'nb_of_points', 'only_integers', 'start', 'steps', 'var', 'xscale'] We see, from the names, that they are information about the curve itself: its color, number of points, and the points themselves. And the points are what we want, to be able to build our own plots directly with Matplotlib. We can get the points with the get_points method (suggestive, isn\u0026rsquo;t it?):\n# the number of values is quite long, so I used a bit of NumPy here # just to show the first three and the last three values of the x axis # and of the y axis in a friendlier way. It is not needed to get the # points themselves, it is only a presentation device for the article with np.printoptions(threshold=10, precision=3, suppress=True): display(np.array(p[0].get_points())) array([[ 0.1 , 0.167, 0.242, ..., 3.89 , 3.943, 4. ], [244.654, 146.637, 101.282, ..., 6.29 , 6.204, 6.116]]) As we can see, the \u0026ldquo;x axis\u0026rdquo; values are in the first list, from 0.1 to 4, as requested when calling the plot method, and the associated y values are in the second list. Let\u0026rsquo;s see the number of elements in each list:\nlen(p[0].get_points()[0]), len(p[0].get_points()[1]) \\(\\displaystyle \\left( 65, \\ 65\\right)\\)\nThe number of points is estimated by SymPy itself. It is possible to customize it, but honestly I never found it necessary, the amounts calculated by SymPy have always given me well-presented plots.\nLet\u0026rsquo;s understand the structure of the object generated by get_points. We have already seen that there are two lists inside the object, but let\u0026rsquo;s look at the type of the object:\ntype(p[0].get_points()) tuple So, we see that it is a tuple and, inside the tuple, there is a list for the x values and another for the y values. We already know everything we need. Let\u0026rsquo;s assign a variable to this object, create a Matplotlib figure, and make our plot:\ndata = p[0].get_points() fig, ax = plt.subplots(figsize=(8, 6), facecolor=(1, 1, 1)) ax.plot(*data) # unpack the tuple OK, we got more or less to what we already had\u0026hellip; so what?\nSo now we can automate the whole creation process. Let\u0026rsquo;s generate a plot with 6 temperatures. If we were to do it via SymPy, it would be a rather tedious procedure, creating each expression and passing them one by one to SymPy\u0026rsquo;s plot method. Now we can do it more quickly, since we know how to extract the data from SymPy.\nLet\u0026rsquo;s start by creating our temperature values. Nobody deserves to do Kelvin math in their head, so let\u0026rsquo;s think of round values in Celsius and let Python do the conversion for us:\ntemperatures_celsius = (-100, 0, 25, 100, 500, 1000) temperatures_kelvin = tuple(t + 273.15 for t in temperatures_celsius) Now, think. We want six curves, we need the data for each one. Let\u0026rsquo;s create an empty list to store this data. And this data needs to be extracted from SymPy expressions with get_points. We could create each expression and extract it, but we don\u0026rsquo;t actually need to create one expression per temperature, we can reuse the same variable, there is no interest in having objects for each expression. Basically, we describe a for loop:\nplots = [] for temperature in temperatures_kelvin: expr = volume.subs({R: gas_constant, n: 1, T: temperature}) p = plot(expr, (P, 0.1, 4), show=False) data = p[0].get_points() plots.append(data) Note that inside the loop we did the same procedure described earlier in the article. But we extracted the data on each pass and appended (append) it to our list. So, our list has 6 data sets, one set of (x, y) points, or rather (P, V), for each temperature:\nlen(plots) \\(\\displaystyle 6\\)\nNow it\u0026rsquo;s time to use the power of Matplotlib. Following the composition we described earlier, let\u0026rsquo;s create a figure and attach axes systems to this figure, each system with one of the curves. And, of course, label the figure and each axis, and add a legend. Full service:\nfig, ax = plt.subplots(figsize=(12, 8), facecolor=(1, 1, 1)) for i, p in enumerate(plots): ax.plot(*p, label=f\u0026#39;T = {temperatures_celsius[i]} °C\u0026#39;, linewidth=3) ax.set_ylim(0, 80) ax.legend(fontsize=14) ax.set_ylabel(\u0026#39;Volume / liter\u0026#39;, fontsize=16) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;, fontsize=16) ax.set_title(\u0026#34;Molar volume - ideal gas\u0026#34;, fontsize=18) plt.show() So satisfying! The procedure can now be extracted to a script, for example, and reproduced for any other case, not necessarily with the ideal gas law but with any other equation with only one variable.\nBut 2D is for the weak, we want it in 3D!\nAs the meme says:\n2D? I wanna see it in 3D! 3D plots # Let\u0026rsquo;s recall objects created at the very beginning:\nideal_gas_law \\(\\displaystyle P V = R T n\\)\nvolume \\(\\displaystyle \\frac{R T n}{P}\\)\nAt the beginning, we fixed the temperatures at some values and generated 2D plots, which are isotherms. But, now, we want the temperature to be one of the axes of the plot. So, the only variable we will fix is the amount of substance in mol, at the value of 1 mol:\nmolar_volume = volume.subs({R: gas_constant, n: 1}) molar_volume \\(\\displaystyle \\frac{0.08205736608096 T}{P}\\)\nAt the beginning, we imported SymPy\u0026rsquo;s plot3d method. The way to use it is very similar to what we saw for plot, except that now we will pass ranges not only for the pressure but also for the temperature:\nplot3d(molar_volume, (P, 0.1, 4), (T, 0, 1000)) Thus, we generated a PVT surface with pressure in the range between 0.1 and 4 atm and temperature in the range from 0 to 1000 K. All the two-dimensional plots you have seen when studying gases are nothing more than slices through planes of surfaces like this one, keeping something constant: the temperature (isotherm plot), the pressure (isobar plot), or the volume (isochore plot).\nNote that SymPy already tries to add some touches to the plot, for example, using a color gradient on the surface, indicating larger values of the z axis with warmer colors, in our case the volume axis. But, we want even more power of customization, so let\u0026rsquo;s extract the data. The procedure is analogous to the one seen before:\np = plot3d(molar_volume, (P, 0.1, 4), (T, 0, 1000), show=False) p \u0026lt;sympy.plotting.plot.Plot at 0x7f46a06d8d30\u0026gt; p[0] \u0026lt;sympy.plotting.plot.SurfaceOver2DRangeSeries at 0x7f46a06d8f40\u0026gt; Note that the object is now different, it is SurfaceOver2DRangeSeries, literally a surface over a range of values in two dimensions (P and T). It makes total sense in our context. Let\u0026rsquo;s look at the public attributes and methods associated with this object:\n[x for x in dir(p[0]) if not x.startswith(\u0026#39;_\u0026#39;)] ['end_x', 'end_y', 'expr', 'get_color_array', 'get_meshes', 'is_2Dline', 'is_3D', 'is_3Dline', 'is_3Dsurface', 'is_contour', 'is_implicit', 'is_line', 'is_parametric', 'nb_of_points_x', 'nb_of_points_y', 'start_x', 'start_y', 'surface_color', 'var_x', 'var_y'] Note that there is no get_points, and it makes sense that there isn\u0026rsquo;t. When we deal with plots of surfaces in 3 dimensions, what we have are pairs of coordinates (x, y) associated with values in z. These pairs build what is usually called a mesh grid. That is why there is a method in NumPy to create meshgrids that you will see in several examples in the Matplotlib documentation on generating 3D surfaces.\nThus, we recognize that what we want is provided by get_meshes:\n# the number of values is quite long, so I used a bit of NumPy here # just to show the first three and the last three values of the matrices # in a friendlier way. It is not needed to get the # values themselves, it is only a presentation device for the article with np.printoptions(threshold=10, precision=3, suppress=True, linewidth=100): display(np.array(p[0].get_meshes())) array([[[ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ], [ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ], [ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ], ..., [ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ], [ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ], [ 0.1 , 0.18 , 0.259, ..., 3.841, 3.92 , 4. ]], [[ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 20.408, 20.408, 20.408, ..., 20.408, 20.408, 20.408], [ 40.816, 40.816, 40.816, ..., 40.816, 40.816, 40.816], ..., [ 959.184, 959.184, 959.184, ..., 959.184, 959.184, 959.184], [ 979.592, 979.592, 979.592, ..., 979.592, 979.592, 979.592], [1000. , 1000. , 1000. , ..., 1000. , 1000. , 1000. ]], [[ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 16.746, 9.325, 6.461, ..., 0.436, 0.427, 0.419], [ 33.493, 18.649, 12.922, ..., 0.872, 0.854, 0.837], ..., [ 787.081, 438.261, 303.677, ..., 20.493, 20.077, 19.677], [ 803.827, 447.586, 310.138, ..., 20.929, 20.504, 20.096], [ 820.574, 456.91 , 316.599, ..., 21.365, 20.931, 20.514]]]) Let\u0026rsquo;s assign a variable:\ndata = p[0].get_meshes() Now we have total control over what to do. For example, didn\u0026rsquo;t quite understand the mesh part? Let\u0026rsquo;s make a plot of just points with the data:\nfig = plt.figure(figsize=(12, 8), facecolor=(1, 1, 1)) ax = fig.add_subplot(111, projection=\u0026#39;3d\u0026#39;) ax.scatter(*data) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;) ax.set_ylabel(\u0026#39;Temperature / K\u0026#39;) ax.set_zlabel(\u0026#39;Volume / l\u0026#39;) plt.show() See how \u0026ldquo;lines\u0026rdquo; are created, sequential values in the pressure x temperature plane, both lines parallel to the pressure axis and parallel to the temperature axis. Notice, in the representation shown above of the values provided by get_meshes, how values repeat. That is the grid (mesh) of paired values (P, T). And for each pair we have an associated volume value V shown on the vertical axis. I hope that helped you understand it better.\nPersonally, the representation I find most useful is the \u0026ldquo;wireframe\u0026rdquo;:\nfig = plt.figure(figsize=(12, 8), facecolor=(1, 1, 1)) ax = fig.add_subplot(111, projection=\u0026#39;3d\u0026#39;) ax.plot_wireframe(*data, rstride=5, cstride=5) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;) ax.set_ylabel(\u0026#39;Temperature / K\u0026#39;) ax.set_zlabel(\u0026#39;Volume / l\u0026#39;) plt.show() Can you recognize the isotherms and the isobars in the plot?\nFinally, we can also create surfaces:\nfig = plt.figure(figsize=(12, 8), facecolor=(1, 1, 1)) ax = fig.add_subplot(111, projection=\u0026#39;3d\u0026#39;) ax.plot_surface(*data) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;) ax.set_ylabel(\u0026#39;Temperature / K\u0026#39;) ax.set_zlabel(\u0026#39;Volume / l\u0026#39;) plt.show() And color it with any colormap available in Matplotlib:\nfig = plt.figure(figsize=(12, 8), facecolor=(1, 1, 1)) ax = fig.add_subplot(111, projection=\u0026#39;3d\u0026#39;) ax.plot_surface(*data, cmap=\u0026#39;rainbow\u0026#39;) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;) ax.set_ylabel(\u0026#39;Temperature / K\u0026#39;) ax.set_zlabel(\u0026#39;Volume / l\u0026#39;) plt.show() And, since we\u0026rsquo;re here, let\u0026rsquo;s make this plot spin. Why? Because I can, the plot is mine and I do what I want ;-)\n%matplotlib notebook fig = plt.figure(figsize=(6, 4), facecolor=(1, 1, 1)) ax = fig.add_subplot(111, projection=\u0026#39;3d\u0026#39;) ax.plot_surface(*data, cmap=\u0026#39;rainbow\u0026#39;) ax.set_xlabel(\u0026#39;Pressure / atm\u0026#39;) ax.set_ylabel(\u0026#39;Temperature / K\u0026#39;) ax.set_zlabel(\u0026#39;Volume / l\u0026#39;) def animate(angle): ax = plt.gca() # gca = get current axis ax.view_init(30, angle) anim = animation.FuncAnimation(fig, animate, frames=361, # 360° rotation interval=1, repeat=False) plt.show() Conclusion # Two of my favorite libraries in a single article. Even though this article is part of the series on SymPy, much was discussed about how Matplotlib builds plots. This knowledge is fundamental to get the most out of everything that is possible in terms of plots. We also saw how SymPy behaves when it interacts with other libraries, and how SymPy\u0026rsquo;s internal instructions are not always passed on to other libraries.\nSee you next time.\n","date":"2021-11-08","externalUrl":null,"permalink":"/2021/11/plots-with-sympy-and-matplotlib/","section":"Blog","summary":"In this article we will see how to make plots with SymPy and Matplotlib. When people talk about SymPy, we usually focus on solving mathematical problems. But many forget that the library has some basic features for building plots, which we will see in this article.","title":"Plots with SymPy and Matplotlib","type":"posts"},{"content":" In this article we will see how to work with rational expressions, exponentials, and logarithms using Python\u0026rsquo;s SymPy library.\nImporting SymPy and creating some symbols # Let\u0026rsquo;s start by importing SymPy and creating some symbols that we will use throughout the article. If you have any trouble with the library\u0026rsquo;s concept of symbols, see the first article in the series about SymPy:\nimport sympy # setup for better outputs in the article, can be ignored sympy.init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.0, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;,) x, y, a, b, c, d, n = sympy.symbols(\u0026#39;x y a b c d n\u0026#39;) Rational expressions # A rational function is any function that can be expressed as a ratio (quotient) of polynomials:\n$$ f(x) = \\frac{P(x)}{Q(x)} $$By default, SymPy does not combine or split rational expressions. For example:\na/b + c/d \\(\\displaystyle \\frac{a}{b} + \\frac{c}{d}\\)\nIf we want to \u0026ldquo;combine\u0026rdquo; the fractions, which by hand we would do with the least common multiple (LCM) procedure, we use the together method:\nsympy.together(_) \\(\\displaystyle \\frac{a d + b c}{b d}\\)\nRemember, the underscore (_) is just a shorthand for reusing a result from a previous cell.\nNow let\u0026rsquo;s see the opposite: a situation where we have a rational expression and would like to write it as simpler fractions, called partial fractions. A situation where we usually do this operation is when solving integrals of rational functions. In this case, we use the apart method:\n(x**2 + x + 4)/(x + 2) \\(\\displaystyle \\frac{x^{2} + x + 4}{x + 2}\\)\nsympy.apart(_) \\(\\displaystyle x - 1 + \\frac{6}{x + 2}\\)\nExponentials and logarithms # Logarithms and exponentials show up in the most diverse mathematical problem contexts. So it is important to know how to use them with SymPy, especially the implicit assumptions the package makes, which can give rise to results that are not always what we expect at first.\nStarting with the notation, in most programming languages and their libraries, log stands for the natural logarithm, which in written work we usually write as ln. Python and SymPy are no different, but SymPy tries to be friendlier and treats ln = log:\nsympy.ln(x) \\(\\displaystyle \\log{\\left(x \\right)}\\)\nNotice that the result shows log(x) even though we wrote ln(x) in the cell. And remember that this is a \u0026ldquo;courtesy\u0026rdquo; of SymPy and not of Python itself. For example, don\u0026rsquo;t try it with the standard math library:\nimport math math.ln(10) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /tmp/ipykernel_16553/2001684872.py in \u0026lt;module\u0026gt; 1 import math 2 ----\u0026gt; 3 math.ln(10) AttributeError: module 'math' has no attribute 'ln' math.log(10) # this works, log being the representation of ln. Not to be confused with log10 \\(\\displaystyle 2.30258509299405\\)\nmath.log10(10) \\(\\displaystyle 1.0\\)\nIn more basic math courses, the following identities are taught for logarithms:\n$$ \\begin{align*} \\log (xy) \u0026= \\log(x) + \\log(y) \\\\ \\log (x^n) \u0026= n \\log(x) \\end{align*} $$However, such identities are not valid if x and y are arbitrary complex numbers, because of the branch point (discontinuity) that a logarithm has in the complex plane. Let\u0026rsquo;s see how SymPy handles such expressions:\nsympy.log(x * y) \\(\\displaystyle \\log{\\left(x y \\right)}\\)\nsympy.log(x**n) \\(\\displaystyle \\log{\\left(x^{n} \\right)}\\)\nNotice that SymPy kept the multiplication and the exponentiation. Let\u0026rsquo;s try to force the expansion with the expand_log method:\nsympy.expand_log(sympy.log(x * y)) \\(\\displaystyle \\log{\\left(x y \\right)}\\)\nsympy.expand_log(sympy.log(x**n)) \\(\\displaystyle \\log{\\left(x^{n} \\right)}\\)\nNotice that it \u0026ldquo;didn\u0026rsquo;t work\u0026rdquo;. Or rather, it worked, because we did not specify any assumptions about x, y, and n. So SymPy makes no implicit assumptions and considers that, in the worst case, they are complex and the expansion cannot be carried out.\nNow, expand_log, like other SymPy methods that we will see in other articles of this series, has a parameter called force that, when True, forces the expansion to be carried out:\nsympy.expand_log(sympy.log(x * y), force=True) \\(\\displaystyle \\log{\\left(x \\right)} + \\log{\\left(y \\right)}\\)\nsympy.expand_log(sympy.log(x**n), force=True) \\(\\displaystyle n \\log{\\left(x \\right)}\\)\nBut in real life, good ideas don\u0026rsquo;t need to be forced. Or, bringing it to a context closer to programming, let\u0026rsquo;s remember the Zen of Python:\nimport this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! Look at the second line: explicit is better than implicit. So, rather than forcing the expansion, it would be better to state the assumptions for each symbol explicitly and, if those assumptions are adequate, the expansion happens naturally.\nThe identities above are valid as long as x and y are positive and n is a real number. So let\u0026rsquo;s redefine those symbols, stating those assumptions explicitly:\nx, y = sympy.symbols(\u0026#39;x y\u0026#39;, positive=True) n = sympy.symbols(\u0026#39;n\u0026#39;, real=True) Let\u0026rsquo;s check whether the expansions are now possible:\nsympy.expand_log(sympy.log(x * y)) \\(\\displaystyle \\log{\\left(x \\right)} + \\log{\\left(y \\right)}\\)\nsympy.expand_log(sympy.log(x**n)) \\(\\displaystyle n \\log{\\left(x \\right)}\\)\nIn my opinion, the second form is much better. Whenever you know the assumptions about each symbol (whether it is always positive, negative, real\u0026hellip;) it is good practice to state them when creating the symbol.\nIf the opposite operation, combining, is desired, there is logcombine:\nsympy.logcombine(sympy.log(x) + sympy.log(y)) \\(\\displaystyle \\log{\\left(x y \\right)}\\)\nsympy.logcombine(n * sympy.log(x)) \\(\\displaystyle \\log{\\left(x^{n} \\right)}\\)\nFinally, let\u0026rsquo;s talk about Euler\u0026rsquo;s number, the base of the natural logarithm. This number has several definitions:\n$$ e \\equiv \\lim_{n \\to \\infty} \\left(1 + \\frac{1}{n}\\right)^n \\equiv \\lim_{x \\to 0} \\left(1 + x\\right)^{\\frac{1}{x}} \\equiv \\sum_{n=0}^{\\infty} \\frac{1}{n!} $$In SymPy it is represented by E. Thus, exp(x) is equivalent to E**x:\nsympy.E \\(\\displaystyle e\\)\nsympy.E**x \\(\\displaystyle e^{x}\\)\nsympy.exp(x) \\(\\displaystyle e^{x}\\)\nWe can get a numerical approximation for E using methods already seen in other articles:\nsympy.E.n() \\(\\displaystyle 2.71828182845905\\)\nLet\u0026rsquo;s check with E that log really represents the natural logarithm:\nsympy.log(sympy.E**2) \\(\\displaystyle 2\\)\nBut what if we want a logarithm in some other base? Just pass the base as the second argument of the log method:\nsympy.log(10**2, 10) \\(\\displaystyle 2\\)\nConclusion # Another article about SymPy. Rational expressions, logarithms, and exponentials show up many times in many concepts, so what we saw in this article will be useful many times in the upcoming articles.\nSee you next time.\n","date":"2021-11-02","externalUrl":null,"permalink":"/2021/11/rational-logarithms-exponentials-sympy/","section":"Blog","summary":"In this article we will see how to work with rational expressions, exponentials, and logarithms using Python’s SymPy library","title":"Rational Expressions, Exponentials, and Logarithms with SymPy","type":"posts"},{"content":" A large part of fundamental math education is devoted to solving equations and systems of equations. In this article we will see how to use SymPy for these tasks quickly and intuitively.\nThe famous quadratic equation # Let\u0026rsquo;s start by importing SymPy and creating a few symbols. We already talked about creating symbols in the first article of this series on SymPy, so, if you have any doubts, take a look there.\nimport sympy # setup for better outputs in the article, can be ignored sympy.init_printing(use_latex=\u0026#39;png\u0026#39;, scale=1.0, order=\u0026#39;grlex\u0026#39;, forecolor=\u0026#39;Black\u0026#39;, backcolor=\u0026#39;White\u0026#39;,) x, y, a, b, c = sympy.symbols(\u0026#39;x y a b c\u0026#39;) Using expressions and solve # Let\u0026rsquo;s now create an expression in the variable x:\nexpr = x**2 + 2*x - 8 As we can see, the expression has degree 2 in x:\nexpr \\(\\displaystyle x^{2} + 2 x - 8\\)\nSo, if we set the expression equal to zero, we get a quadratic equation, and we can solve it. That is, find the values of x for which the equation equals zero, the roots of the equation. For that, we use the solve method with the signature solve(expr, var), where var is the variable for which we want to solve the equation:\nsympy.solve(expr, x) \\(\\displaystyle \\left[ -4, \\ 2\\right]\\)\nSo, we see that the roots of the equation x2+2x-8=0 are -4 and 2.\nNow, the great power of SymPy is being a symbolic CAS. So, we can get the general expressions for the roots of a quadratic equation written in the general form ax2+bx+c=0:\nsympy.solve( a*x**2 + b*x + c, x) \\(\\displaystyle \\left[ \\frac{- b - \\sqrt{- 4 a c + b^{2}}}{2 a}, \\ \\frac{- b + \\sqrt{- 4 a c + b^{2}}}{2 a}\\right]\\)\nWell, that is nothing more than the famous quadratic formula. The formula usually presented in books contains the symbol ± to indicate that there are two roots. Above we have the two roots in a more explicit form.\nWe saw earlier that the solution is presented as a list with each root. Let\u0026rsquo;s, then, look at each solution separately by accessing each item of the list in the usual Python way:\nsolutions = sympy.solve( a*x**2 + b*x + c, x) solutions[0] \\(\\displaystyle \\frac{- b - \\sqrt{- 4 a c + b^{2}}}{2 a}\\)\nsolutions[1] \\(\\displaystyle \\frac{- b + \\sqrt{- 4 a c + b^{2}}}{2 a}\\)\nSince lists are iterable, we can write a loop substituting a, b and c with the values used in the first expression of this article and check that the roots are, indeed, -4 and 2:\nroots = [] for solution in solutions: roots.append(solution.subs({\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2, \u0026#39;c\u0026#39;: -8})) roots \\(\\displaystyle \\left[ -4, \\ 2\\right]\\)\nUsing equalities and solve # The previous approach is perfectly correct and is widely used in scripts and tutorials you find on the internet. However, there is another way to approach the problem, which uses an abstraction of equality that exists in SymPy and that, personally, I find more interesting because it is more semantic, closer to what we actually want to represent mathematically.\nSymPy has the class Equality, which is meant to express the equality between two objects. There is the abbreviation Eq, which makes code easier to write. So, the same expression as before, which we implicitly treated as an equation only when we ran solve(expr, x), could be recognized more explicitly, directly as an equation, with:\nequation = sympy.Eq(x**2 + 2*x - 8, 0) equation \\(\\displaystyle x^{2} + 2 x - 8 = 0\\)\nNotice that now we actually wrote it as an equation, with a left-hand side and a right-hand side. The right-hand side, in this case, being the number zero. Certainly this form is easier to understand.\nAnd, of course, we can pass this equation to solve and get the same roots:\nsympy.solve(equation, x) \\(\\displaystyle \\left[ -4, \\ 2\\right]\\)\nThe right-hand side does not necessarily have to be zero. The same equation could be represented by:\nsympy.Eq(x**2 + 2*x, 8) \\(\\displaystyle x^{2} + 2 x = 8\\)\nWhich, obviously, would result in the same roots when passed to solve:\nsympy.solve(sympy.Eq(x**2 + 2*x, 8), x) \\(\\displaystyle \\left[ -4, \\ 2\\right]\\)\nSystems of equations # We have seen that we can solve an equation, but what about a system? We easily can. The first way is to pass an iterable of expressions to solve and, also, the variables of the system. Suppose a system formed by the equations x + y = 3 and 3x - 2y = 0. We could write:\nsympy.solve((x + y - 3, 3*x - 2*y), (x, y)) \\(\\displaystyle \\left\\{ x : \\frac{6}{5}, \\ y : \\frac{9}{5}\\right\\}\\)\nWe could also be more explicit by creating each equation with Eq and passing them to solve:\neq1 = sympy.Eq(x + y, 3) eq2 = sympy.Eq(3*x - 2*y, 0) sympy.solve((eq1, eq2), (x, y)) \\(\\displaystyle \\left\\{ x : \\frac{6}{5}, \\ y : \\frac{9}{5}\\right\\}\\)\nBeware of floats! # Notice that the result of the system came out as a dictionary. And we can actually substitute it into any equation to see the result:\neq1.subs({x: 6/5, y: 9/5}) \\(\\displaystyle \\text{True}\\)\nWhat does the result True mean? It means that the equality of the equation is satisfied. Look at the representation of eq1:\neq1 \\(\\displaystyle x + y = 3\\)\nSo, True means that, when the given values replace x and y in the equation, the equality is satisfied, it is true.\nWell, one would expect that the same substitution in eq2 would return True, right?\neq2.subs({x: 6/5, y: 9/5}) \\(\\displaystyle \\text{False}\\)\nHuh\u0026hellip;?!\nBeware of floats! In the first article about SymPy we already covered a bit of this and saw how we can represent fractions in SymPy. See, when you write 6/5 in Python, what we get is:\n6/5 \\(\\displaystyle 1.2\\)\nThat is, a float. And operations with floats involve approximations, as we saw in the article already cited. So, it pays to convert them into SymPy\u0026rsquo;s fraction representations:\neq2.subs({x: sympy.S(\u0026#39;6/5\u0026#39;), y: sympy.S(\u0026#39;9/5\u0026#39;)}) \\(\\displaystyle \\text{True}\\)\nNow we have the equality as valid.\nCompleting the square # Completing the square is a technique to convert a polynomial in the form ax2+bx+c to the form a(x-h)2+k for some values of h and k. It is used in many teaching contexts, the best known being the derivation of the quadratic formula.\nIn the following animation, we see that h=-b ⁄ (2a) and k= c-(b2 ⁄ 4a):\nWe then need to create two new symbols:\nh, k = sympy.symbols(\u0026#39;h k\u0026#39;) We can, just as a teaching resource, express the equality in SymPy:\nsquare = sympy.Eq(a * x**2 + b*x + c, a * (x - h)**2 + k) square \\(\\displaystyle a x^{2} + b x + c = a \\left(- h + x\\right)^{2} + k\\)\nLet\u0026rsquo;s now take any quadratic equation and try to find the values of h and k. For example: x2-4x+7 = (x-h)2+k\nsquare_example = sympy.Eq(x**2 - 4*x + 7, (x - h)**2 + k) sympy.solve(square_example, (h, k)) \\(\\displaystyle \\left\\{ h : 2, \\ k : 3\\right\\}\\)\nGetting to know the Equality object # Let\u0026rsquo;s take the opportunity to get to know the Equality object better with this example. So far, we have used it as a way to represent a mathematical equation, but we have already seen that the return can also be a boolean, True or False. Let\u0026rsquo;s understand.\nFirst, let\u0026rsquo;s substitute the values found for h and k into our object:\nsquare_example.subs({h: 2, k: 3}) \\(\\displaystyle x^{2} - 4 x + 7 = \\left(x - 2\\right)^{2} + 3\\)\nWe can get each side of the equality with the attributes rhs (right-hand side) and lhs (left-hand side):\nsquare_example.subs({h: 2, k: 3}).rhs \\(\\displaystyle \\left(x - 2\\right)^{2} + 3\\)\nsquare_example.subs({h: 2, k: 3}).lhs \\(\\displaystyle x^{2} - 4 x + 7\\)\nWe can expand the square on the right-hand side:\nsquare_example.subs({h: 2, k: 3}).rhs.expand() \\(\\displaystyle x^{2} - 4 x + 7\\)\nWell, this shows that the two sides are indeed equal, right? So, the following equality comparison should be true:\nsquare_example.subs({h: 2, k: 3}).lhs == square_example.subs({h: 2, k: 3}).rhs False But no\u0026hellip; why? According to the SymPy documentation, the library checks for structural equality, not mathematical equivalence. For details, see this part of the SymPy documentation and this FAQ in the SymPy repository.\nFrom what we have seen, if the comparison is between the left-hand side and the expansion of the right-hand side, the result should be True:\nsquare_example.subs({h: 2, k: 3}).lhs == square_example.subs({h: 2, k: 3}).rhs.expand() True Now it works. And everything could be summed up in the following line, which tells SymPy to carry out the possible expansions and check the equality:\nsquare_example.subs({h: 2, k: 3}).expand() \\(\\displaystyle \\text{True}\\)\nConclusion and more articles about SymPy # The temptation to solve every single equation that shows up in front of you with SymPy must now be great. Systems, even more so! But see how important it is to have a good mathematical foundation to understand what can be done with the library. Don\u0026rsquo;t go skipping math class (or do, but then study from the books ;-) ).\nSee you next time.\n","date":"2021-10-08","externalUrl":null,"permalink":"/2021/10/solving-equations-with-sympy/","section":"Blog","summary":"A large part of fundamental math education is devoted to solving equations and systems of equations. In this article we will see how to use SymPy for these tasks quickly and intuitively.","title":"Solving Equations with SymPy","type":"posts"},{"content":" Many people consider mathematics, physics, and related subjects difficult, usually because of the traditional approach found in schools. But the truth is that, when presented in a more suitable way, they are amazing fields that, once mastered, give us a different view of the world. And computers can help with the more complex and tedious steps of arithmetic manipulation. In this article, we will start exploring the SymPy library, taking the chance to do a good review of fundamental mathematical concepts.\nIntroduction and installation of SymPy # A computer algebra system (CAS) can be used for calculations with complicated mathematical expressions, solving equations, simulations, among other applications.\nThere are free systems such as SymPy and Octave, and commercial ones such as Maple, MATLAB, and Mathematica. In essence, they all offer the same features, so the choice comes down to technical aspects and cost-benefit, especially with respect to integration with other tools. In this sense, SymPy has a big advantage, since Python is widely used in many contexts, so its functionality can easily be expanded. In fact, on the SymPy project website, these are the highlighted points:\nfree, under the BSD license based on Python lightweight, with mpmath as its only dependency it is a library, so it can be used in other applications and easily extended And, the main reason to choose SymPy: you can check your answers to the calculus problem sets from college, or to those exams by uncreative professors, full of calculations and with no real application or context ;-) Just don\u0026rsquo;t say I wrote this.\nThat is the face of someone tackling one of those problem sets:\nNazaré confused by calculus And if you are a professor and the shoe fits, calm down and breathe. Of course students need to learn calculus; if they don\u0026rsquo;t know the basics they won\u0026rsquo;t even know how to use the package, since computers don\u0026rsquo;t work miracles, they return exactly what was asked. The criticism is about applying this knowledge to real cases. Create projects where the calculations are applied, create problem solvers and not human calculators.\nSymPy is a symbolic computer algebra system. This means that numbers and operations are represented symbolically, which makes it possible to obtain exact results. Let\u0026rsquo;s understand this better.\nConsider the number \\(\\sqrt{2}\\). In SymPy, this number is represented by the object Pow(2, 1/2), whereas in numerical CAS, such as Octave, it is represented as an approximation 1.41421356237310 (a float). So what? Well, depending on the application, there is no problem in having this approximation, but it can lead to problems. Let\u0026rsquo;s see an example.\nIf we use the sqrt function from Python\u0026rsquo;s math package, we also get the root stored as a float:\nimport math math.sqrt(2) 1.4142135623730951 Well, we know that if we square \\(\\sqrt{2}\\), we should get the number 2, right? Let\u0026rsquo;s see:\nmath.sqrt(2)**2 2.0000000000000004 Did you see what happened? There is a stray \u0026ldquo;4\u0026rdquo; there that, in a very simplified way, arises from the fact that the representation is an approximation.\nAnd what is the consequence of this? We could list a few, but I will focus on one that catches many beginners by surprise. It is very common to use comparisons with == in conditionals or in function tests. These comparisons run a serious risk of failing when they involve floats:\nmath.sqrt(2)**2 == 2 False What to do in these cases? Well, if the context requires continuing to use numerical approximations and floats, a simple but not always reliable way is to compare with a tolerance that is acceptable in the context. For example, say that an error of 1 in 10,000 is acceptable in a given context. Then the test could be written as:\nabs(math.sqrt(2)**2 - 2) \u0026lt; 1E-4 True At some point I will write articles covering in more depth the care needed when working with floats, but for now here is a reading recommendation: floating point arithmetic from the Python documentation itself.\nLet\u0026rsquo;s see how SymPy behaves in this simple example.\nFirst, installation can be done with a simple pip install sympy. If you use Anaconda, which we have already written about here, SymPy is already installed. Importing it is simple:\nimport sympy Let\u0026rsquo;s see how the square root of 2 is represented:\nsympy.sqrt(2) \\(\\displaystyle \\sqrt{2}\\)\nNote that the symbol \\(\\sqrt{2}\\) appeared, not an approximation in the form of a float. Let\u0026rsquo;s do the squaring operation:\nsympy.sqrt(2)**2 \\(\\displaystyle 2\\)\nWe got the exact result 2. We can even verify that it is really the number 2 with the comparison:\nsympy.sqrt(2)**2 == 2 True And, since the expression is treated symbolically, it can be simplified symbolically. Remember that \\(\\sqrt{8} = 2\\sqrt{2}\\)? Then:\nsympy.sqrt(8) \\(\\displaystyle 2 \\sqrt{2}\\)\nSymPy took care of this simplification for you. And, what is more, the relationship becomes explicit symbolically, whereas if you compute \\(\\sqrt{8}\\) on your calculator or with a numerical package you will hardly notice that the result corresponds to \\(2\\sqrt{2}\\):\nmath.sqrt(8) 2.8284271247461903 Let\u0026rsquo;s now see the basics of mathematics with SymPy.\nBasic mathematics with SymPy # Let\u0026rsquo;s understand how SymPy represents mathematical objects and simple operations. But first, let\u0026rsquo;s remember that in Python there are two main numeric types: integers and floats.\n3 # int 3 3.0 # float 3.0 Floats are approximate representations of real numbers, up to the 16th decimal place. When we do a division, even of integers, the result is a float:\n1/1 1.0 1/7 0.14285714285714285 The so-called floor division, represented by two slashes, gives the integer part of the division:\n1//7 0 While the % sign, known as modulo, gives the remainder of the division:\n1 % 7 1 Rational numbers # Now, we know that 1/7 belongs to the set of rational numbers and its decimal representation is infinitely long. For an exact representation, we need SymPy\u0026rsquo;s sympify function:\nsympy.sympify(\u0026#39;1/7\u0026#39;) \\(\\displaystyle \\frac{1}{7}\\)\nWhich can be called by the short name S:\nsympy.S(\u0026#39;1/7\u0026#39;) \\(\\displaystyle \\frac{1}{7}\\)\nNote how the fraction was passed to the function, as a string. Let\u0026rsquo;s see the type of the generated object:\ntype(sympy.S(\u0026#39;1/7\u0026#39;)) sympy.core.numbers.Rational So, the same fraction could be created as:\nsympy.Rational(1, 7) \\(\\displaystyle \\frac{1}{7}\\)\nNote that in the form above we passed the numerator and the denominator as integers. We can do operations between rationals:\nsympy.Rational(1, 2) + sympy.S(\u0026#39;1/3\u0026#39;) \\(\\displaystyle \\frac{5}{6}\\)\nA SymPy object divided by an integer results in a SymPy object:\nsympy.S(\u0026#39;1\u0026#39;)/7 \\(\\displaystyle \\frac{1}{7}\\)\nObtaining numerical approximations # When solving problems, it is better to leave obtaining the numerical representation until the end. To get the numerical approximation of a SymPy object as a float, we use the evalf() method. Let\u0026rsquo;s see an example with the number \\(\\pi\\):\nsympy.pi \\(\\displaystyle \\pi\\)\ntype(sympy.pi) sympy.core.numbers.Pi Note that it is a SymPy-specific type, since the library has some built-in numbers. We will see some of them over time. Let\u0026rsquo;s see the representation as a float:\nsympy.pi.evalf() \\(\\displaystyle 3.14159265358979\\)\ntype(sympy.pi.evalf()) sympy.core.numbers.Float Note above that SymPy has its own Float class (the same goes for integers and other kinds of numbers), not using Python\u0026rsquo;s standard float. So, be careful when mixing standard Python types and SymPy types. When in doubt, apply the sympify function already shown to make sure everything is a SymPy object.\nA short way to call the evalf method is simply to use the n method:\nsympy.pi.n() \\(\\displaystyle 3.14159265358979\\)\nIt is possible to request more decimal places. For example, if we want 50 decimal places for the number \\(\\pi\\):\nsympy.pi.n(50) \\(\\displaystyle 3.1415926535897932384626433832795028841971693993751\\)\nThis is possible because SymPy uses the mpmath library under the hood, which allows calculations with arbitrary-precision arithmetic.\nThere is also a global function N to obtain a numerical representation of SymPy objects:\nsympy.N(sympy.pi) \\(\\displaystyle 3.14159265358979\\)\nSince this is symbolic mathematics, we can use, for example, the notion of infinity, which has the curious symbol oo:\nsympy.oo \\(\\displaystyle \\infty\\)\nsympy.oo \u0026gt; 100 \\(\\displaystyle \\text{True}\\)\nSymbols # Since we are talking about symbolic algebra, there is nothing better than seeing how to create symbols with SymPy. From Python, we know that if we use a symbol not previously defined, a NameError will be raised:\nx + 2 --------------------------------------------------------------------------- NameError Traceback (most recent call last) /tmp/ipykernel_342/917526016.py in \u0026lt;module\u0026gt; ----\u0026gt; 1 x + 2 NameError: name 'x' is not defined Once we define x, it can be used. And, in Python, there is no need to declare the type of x:\nx = 2 x + 2 4 The point is that we would like x to be recognized as a symbol. Think of the x from middle school math classes, from the famous \u0026ldquo;find x\u0026rdquo; questions:\nClassic \u0026lsquo;find x\u0026rsquo; Hehe\u0026hellip;\nThat is, we want x with the classic meaning of a mathematical variable. We can do it as follows:\nx = sympy.Symbol(\u0026#39;x\u0026#39;) x + 2 \\(\\displaystyle x + 2\\)\nNote that it now appears as a mathematical expression. We can create several symbols at the same time; notice that now the function is symbols:\ny, z = sympy.symbols(\u0026#39;y z\u0026#39;) x + y + z - 5 \\(\\displaystyle x + y + z - 5\\)\nSymPy allows creating symbols that follow a pattern. For example, in the study of polynomials it is very common to represent all the coefficients with the letter a, changing only the subscript number. This can be created with a notation very similar to that of Python sequence slices, in which the last number is exclusive:\na1, a2, a3, a4 = sympy.symbols(\u0026#39;a1:5\u0026#39;) a4 * x**4 + a3 * x**3 + a2 * x**2 + a1 * x + 2 \\(\\displaystyle a_{1} x + a_{2} x^{2} + a_{3} x^{3} + a_{4} x^{4} + 2\\)\nNote that by default SymPy presents the polynomial from the lowest degree in x to the highest, leaving the \\(x^0\\) term for last. It is possible to change the order by changing the monomial order setting of the init_printing function, which configures how SymPy expressions are displayed. Let\u0026rsquo;s see:\nsympy.init_printing(order=\u0026#39;grlex\u0026#39;) a4 * x**4 + a3 * x**3 + a2 * x**2 + a1 * x + 2 \\(\\displaystyle a_{4} x^{4} + a_{3} x^{3} + a_{2} x^{2} + a_{1} x + 2\\)\ngrlex comes from graded lexicographic order, going from the highest degree to the lowest, which is more usual in mathematical texts and the like. To return to SymPy\u0026rsquo;s default, we use the lex option:\nsympy.init_printing(order=\u0026#39;lex\u0026#39;) a4 * x**4 + a3 * x**3 + a2 * x**2 + a1 * x + 2 \\(\\displaystyle a_{1} x + a_{2} x^{2} + a_{3} x^{3} + a_{4} x^{4} + 2\\)\nAccording to the SymPy documentation, some letters are reserved and should be avoided as symbols: I, E, N, S, O, Q, and C. We have already seen N and S. I and E represent the imaginary number \\(i \\equiv \\sqrt{-1}\\) and the base of the natural logarithm, respectively:\nsympy.I \\(\\displaystyle i\\)\nsympy.E \\(\\displaystyle e\\)\nThe O is used for big-O notation:\nsympy.O sympy.series.order.Order sympy.O(x**2) * x \\(\\displaystyle O\\left(x^{3}\\right)\\)\nC and Q have more specific uses, which are beyond the scope of this introductory material, but can be seen in the documentation already cited.\nExpressions # Expressions can be created by combining symbols with mathematical operations and other functions:\nexpr = 2 * x + 3 * x - sympy.sin(x) - 3 * x + 42 expr \\(\\displaystyle 2 x - \\sin{\\left(x \\right)} + 42\\)\nNote that the expression was already presented simplified, since this is a simple case of addition. In more elaborate cases, SymPy is not proactive about simplification, given that the user may want the raw expression, without manipulations. But it is possible to request simplification.\nSimplifying expressions and obtaining coefficients # There is a specific function for simplification:\n(x + x * y) / x \\(\\displaystyle \\frac{x y + x}{x}\\)\nsympy.simplify(_) \\(\\displaystyle y + 1\\)\nThe underscore _ takes the result of the previous cell and passes it to simplify.\nBack to our expression expr, it is possible to check the coefficients of the terms in x with the coeff method, treating the expression as a polynomial in x:\nexpr.coeff(x) \\(\\displaystyle 2\\)\nSince the expression is of order 1 in x, there is, for example, no coefficient for a term of order 2 or, better put, that coefficient is zero:\nexpr.coeff(x, 2) \\(\\displaystyle 0\\)\nIf we want the 42 term in the expression, we just ask for the coefficient of the x^0 term:\nexpr.coeff(x, 0) \\(\\displaystyle 42\\)\nIt is also possible to check the coefficients relative to other terms; just treat them as if they were the polynomial\u0026rsquo;s variable:\nexpr.coeff(sympy.sin(x)) \\(\\displaystyle -1\\)\nFactoring and expanding (distributive property) # There are functions for many usual mathematical operations on expressions. For example, the factoring operation:\nsympy.factor(x**2 - 2*x - 8) \\(\\displaystyle \\left(x - 4\\right) \\left(x + 2\\right)\\)\nAnd its opposite, expansion or distributive multiplication:\nsympy.expand( (x - 4)*(x + 2) ) \\(\\displaystyle x^{2} - 2 x - 8\\)\nIt is also possible to expand trigonometric expressions. For example:\nsympy.cos(x + y) \\(\\displaystyle \\cos{\\left(x + y \\right)}\\)\nsympy.expand(_, trig=True) \\(\\displaystyle - \\sin{\\left(x \\right)} \\sin{\\left(y \\right)} + \\cos{\\left(x \\right)} \\cos{\\left(y \\right)}\\)\nThe collect function # When symbols are used as coefficients, SymPy does not group them. But we can use the collect function, which additively collects terms with respect to a power. To understand, let\u0026rsquo;s create two symbols, a and b, and create an expression with them:\na, b = sympy.symbols(\u0026#39;a b\u0026#39;) expr = x**2 + x*b + a*x + a*b expr \\(\\displaystyle a b + a x + b x + x^{2}\\)\nLet\u0026rsquo;s then pass the expression to the collect function, telling it to group the coefficients of the x^1 term:\nsympy.collect( expr, x ) \\(\\displaystyle a b + x^{2} + x \\left(a + b\\right)\\)\nNote that the expression was written so as to make explicit that the coefficient of the x term is (a + b). This can be useful in some contexts, especially for presentation.\nHowever, even without writing it explicitly, SymPy already recognizes the coefficient internally as (a + b), as can be seen with the coeff method:\nexpr.coeff(x, 1) \\(\\displaystyle a + b\\)\nSubstituting values in an expression # Something of great interest in the context of mathematical expressions is being able to substitute values to check the value of the expression after such a substitution. Let\u0026rsquo;s create a new expression:\nexpr = sympy.sin(x) + sympy.cos(y) expr \\(\\displaystyle \\sin{\\left(x \\right)} + \\cos{\\left(y \\right)}\\)\nWith the subs method, we can pass a dictionary with the desired values for each symbol:\nexpr.subs({x: 1, y:2}) \\(\\displaystyle \\cos{\\left(2 \\right)} + \\sin{\\left(1 \\right)}\\)\nNote that the return was still symbolic, but we can see a float approximation using the n method seen earlier:\nexpr.subs({x: 1, y:2}).n() \\(\\displaystyle 0.425324148260754\\)\nIf the result of the substitution is exact, the return will be presented directly. For example, we know that \\(\\sin(\\pi) = 0\\) and \\(\\cos(0) = 1\\). So the sum would be 1. Let\u0026rsquo;s see:\nexpr.subs({x: sympy.pi, y:0}) \\(\\displaystyle 1\\)\nThe same goes for other special angles, which have symbolic representations. For example:\nexpr.subs({x: sympy.pi/2, y:sympy.pi/4}) \\(\\displaystyle \\frac{\\sqrt{2}}{2} + 1\\)\nIf you did not understand, review the special angles with the following image of the unit circle:\nSpecial angles on the unit circle If we request a substitution with already approximated values, we get the following kind of result:\nexpr.subs({x: sympy.pi/2.5, y:sympy.pi/4.3}) \\(\\displaystyle \\cos{\\left(0.232558139534884 \\pi \\right)} + \\sin{\\left(0.4 \\pi \\right)}\\)\nConclusion and more articles about SymPy # In this article we saw the basics of SymPy, and I hope it was possible to see the power of this library. This is only the first in a series of articles about the library.\nUntil next time.\n","date":"2021-10-02","externalUrl":null,"permalink":"/2021/10/introduction-to-sympy/","section":"Blog","summary":"In this article we take an introductory look at SymPy, one of the most powerful mathematical packages for the Python language","title":"Introduction to SymPy - Calculations with Python","type":"posts"},{"content":"This article describes how developers can contribute pull requests to an upstream repository and how the owners of upstream repositories can accept such pull requests. We will go through the whole workflow involving forks and pull requests popularized by the GitHub platform and equally valid on similar platforms such as GitLab.\nIntroduction # Every project has a main development branch to which developers regularly push commits. Usually, this branch is named master or main, but some projects adopt a develop or trunk branch, or some other one, for day-to-day development activities. This main development branch will be called the main development branch throughout the article to make the text more general. However, in the diagrams and in the examples involving commands, we will use master as the example of this branch to make it easier to understand.\nThe following terms are used in the command examples and in the diagrams of this article:\nGITHUB: github.com or the domain/hostname of your private GitHub Enterprise system USER or CONTRIBUTOR: the user who forks an upstream repository, creates pull requests, and sends them to the upstream repository. UPSTREAM-OWNER: owner of the upstream repository. This is the name of the user or organization that accepts the pull requests to the upstream repository. REPO: name of the repository. FILES: one or more files staged for a commit. TOPIC-BRANCH: branch where the contributor develops a specific feature. Referred to in the text as the topic branch. Obviously, these terms must be replaced by appropriate values when used in a real situation.\nAfter this introductory stage of the article, the terms will no longer be italicized, to make the reading more fluid.\nBeginners should always remember that a branch is not a container of commits, but a pointer that points to a commit in the history.\nIn the figure below, the master branch points to C2, the most recent commit of a sequence of three:\nThe master branch is a pointer to commit C2, the most recent in the history. When a new commit is made on a branch, the pointer simply moves to point to the last commit on the branch.\nAfter a new commit, the master pointer advances to C3. A branch is merely a pointer to the tip of a series of commits. With this in mind, complex operations such as rebase and fast-forward merges become easier to understand and use. If you are not familiar with these operations, no problem, they will be explained at the right time.\nThe next section is a quick guide to the most used commands. The remaining sections go deeper into some more specific but still important aspects of the workflow.\nQuick guide # Here we will see a brief summary of all the commands used in the article to create and merge pull requests. Use this section as a quick reference guide.\nCreating a pull request # Fork the upstream and clone your fork git clone https://GITHUB/USER/REPO.git cd REPO git remote add upstream https://GITHUB/UPSTREAM-OWNER/REPO.git git remote -v Work on a new topic branch. git checkout -b TOPIC-BRANCH git add FILES git commit git push origin TOPIC-BRANCH To create the pull request, go to your fork on GitHub, switch to the topic branch, and click \u0026lsquo;Compare \u0026amp; pull request\u0026rsquo;.\nKeep your fork\u0026rsquo;s main development branch up to date with the upstream\u0026rsquo;s.\ngit checkout master git pull upstream master git push origin master Add files that were left out of the last commit (optional). git add FILES git commit --amend Rebase the topic branch onto the main development branch (optional). git checkout TOPIC-BRANCH git rebase master Edit, for example, the last 3 commits on the topic branch (optional). git checkout TOPIC-BRANCH git rebase -i HEAD~3 Force push rebased/edited commits to the pull request (optional). git push -f origin TOPIC-BRANCH Delete the topic branch after the pull request is merged. git checkout master git branch -D TOPIC-BRANCH git push -d origin TOPIC-BRANCH Merge pull request (without merge commit) # Clone the upstream repository. git clone https://GITHUB/UPSTREAM-OWNER/REPO.git cd REPO Keep the local main development branch up to date. git checkout master git pull Pull the changes of the pull request into a temporary branch. git checkout -b pr git pull https://GITHUB/CONTRIBUTOR/REPO.git TOPIC-BRANCH If the command above does not create a commit for the merge, ignore this comment and follow the instructions after it. If the command above did create a commit for the merge, first know that there is no problem in this situation. However, if you do not want to have this specific merge commit, choose one of the following alternatives:\nAsk the contributor to rebase the topic branch onto the main development branch. After that is done, delete the temporary branch, and start over, pulling the changes again. (Commands: git checkout master; git branch -D pr)\nGo to the pull request page on GitHub, click the menu next to \u0026lsquo;Merge pull request\u0026rsquo;, select \u0026lsquo;Rebase and merge\u0026rsquo;, and click \u0026lsquo;Rebase and merge\u0026rsquo;\nRebase the topic branch onto the main development branch. Then, follow the commands below. Then, close the pull request manually on GitHub. (Command: git rebase master)\nMerge the pull request and delete the temporary branch.\ngit checkout master git merge pr git branch -d pr Push the updated main development branch to the upstream repository. git push origin master Merge pull request (with merge commit) # Clone the upstream repository. git clone https://GITHUB/UPSTREAM-OWNER/REPO.git cd REPO Keep the main development branch up to date. git checkout master git pull Pull the changes of the pull request into a temporary branch. git checkout -b pr git pull https://GITHUB/CONTRIBUTOR/REPO.git TOPIC-BRANCH Merge the pull request and delete the temporary branch. git checkout master git merge --no-ff pr git branch -d pr Push the updated main development branch to the upstream repository. git push origin master The sections Creating Pull Requests and Merging a Pull Request elaborate on these commands in detail.\nGit basics # Important concepts # If you are a complete beginner, it is worth briefly reviewing what Git is and how it works at a basic level. Users who already know what the working directory, the staging area, and the Git directory (the repository itself) are can skip to the next section.\nGit treats its data as a set of snapshots of a miniature filesystem. Every time you commit, or save the state of your project in Git, it basically takes a \u0026ldquo;picture\u0026rdquo; (snapshot) of all your files and stores a reference to that set of files. To be efficient, if files have not changed, Git does not store them again, just a link to the file already stored. Seen another way, Git treats its data as a stream of file states.\nEverything in Git goes through a checksum before being stored and is referenced by that checksum. This means it is impossible to change the contents of any file or folder without Git detecting it. You will not lose information during transfer and will not receive a corrupted file without Git being able to detect it.\nThe mechanism Git uses for this verification is called a SHA-1 hash. This is a sequence of 40 characters made up of hexadecimal characters (0-9 a-f) and is computed from the contents of a file or directory structure in Git. A SHA-1 hash looks like this:\n24b9da6552252987aa493b52f8696cd6d3b00373 Everything is stored in the database by the content hash value and not by the file name.\nYour files can be in three main states: committed, modified, and staged. Committed means the data is safely stored in your local database. \u0026ldquo;Modified\u0026rdquo; means you changed the file but have not yet committed it to your database. \u0026ldquo;Staged\u0026rdquo; means you marked the current version of a modified file to go into your next commit.\nThis leads us to three main sections of a project: the Git directory (which is the repository itself), the working directory, and the staging area.\nThe figure below shows the three sections and the operations that move files from one to another:\nThe three sections of a Git project: checkout brings files from the repository to the working directory, staging takes changes to the staging area, and commit records them in the repository. The Git directory is where Git stores the metadata and the object database of your project. This is the most important part, and it is what is copied when you clone a repository from another computer.\nThe working directory is a simple copy of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.\nThe staging area is a file, generally contained in your Git directory, that stores information about what will go into your next commit. It is sometimes referred to as the \u0026ldquo;index\u0026rdquo;.\nThe basic Git workflow is:\n1 Modify You modify files in your working directory. 2 Stage You stage the files, adding snapshots of them to your staging area. 3 Commit You commit, which takes the files as they are in the staging area and stores snapshots of them (states) permanently in the Git directory. If a particular version of a file is in the Git directory, it is considered committed. If it has been modified but was added to the staging area, it is considered staged. And if it was changed after being checked out but has not been staged, it is considered modified.\nRecording changes # Suppose you have a project in a Git repository. Each time the project reaches a state you want to record, you need to commit the changes you made.\nRemember that each file in your working directory can be in one of two states: tracked and untracked. Tracked files are files that were included in the last snapshot; they can be unmodified, modified, or staged. In short, tracked files are the files Git knows about.\nUntracked files are everything else: any files in your working directory that were not included in your last snapshot and are not in the staging area. When you first clone a repository, all your files will be tracked and unmodified, since Git has just obtained them and you have not yet made any edits.\nAs soon as you edit some files, Git considers them modified, because there have been edits since your last commit. Then, you stage the edited files and commit your changes. And the cycle repeats.\nThe figure below summarizes this file lifecycle:\nThe lifecycle of a file in Git: untracked, unmodified, modified, and staged. The main tool for determining which files are in which state is the git status command.\nThat was a review of the basics of how Git works. Knowing this, the rest of the article should flow naturally.\nCreating Pull Requests # This section is for developers who want to contribute new commits to the upstream repository from their personal forks.\nFork and Clone # On GitHub, fork the upstream repository to your personal account.\nThen, clone your fork from your personal GitHub account to your local system and set the URL of the upstream repository as a remote repository named upstream.\ngit clone https://GITHUB/USER/REPO.git cd REPO git remote add upstream https://GITHUB/UPSTREAM-OWNER/REPO.git git remote -v Now the remote repository named upstream points to the upstream repository and the one named origin points to your fork.\nWork on the Pull Request # Work on the new pull request in a new topic branch and commit to your fork. Remember to use a meaningful name instead of TOPIC-BRANCH in the following commands.\ngit checkout -b TOPIC-BRANCH git add FILES git commit git push origin TOPIC-BRANCH Create the pull request through the GitHub web interface with the following steps:\nGo to your fork on GitHub Switch to the topic branch Click Compare \u0026amp; pull request Click Create pull request Wait until an upstream developer reviews and merges your pull request.\nIf there are reviewer comments that must be taken into account and lead to necessary changes in the code, keep working on your branch, making commits and, optionally, doing rebases, amends, squashes, and all the usual operations. Push your changes to the topic branch of your origin fork. Such changes automatically become available in the pull request.\nIn this fork and pull request workflow, a contributor should never make commits to the main development branch of their personal fork. This way, it is very simple to keep that main branch in sync with the main branch of the upstream repository. This will be explained next.\nIf a reviewer of your pull request pushed commits to your PR branch, you must update your local branch with those commits:\ngit checkout TOPIC-BRANCH git pull origin TOPIC-BRANCH If, in the time between the creation and the review of the pull request, there are commits on the upstream\u0026rsquo;s main branch that lead to conflicts with the pull request, you will see a warning from GitHub in your Pull Requests tab about it and the reviewers may ask you to modify your pull request to resolve these conflicts:\ngit checkout TOPIC-BRANCH git pull upstream master # resolve the conflicts git push origin TOPIC-BRANCH Keep your fork up to date # As new pull requests are merged into the upstream\u0026rsquo;s main development branch, the main development branch of your fork starts to fall behind, getting out of date.\nThe following commands show how to keep your fork\u0026rsquo;s main branch up to date with new commits from the main branch of the upstream repository.\ngit checkout master git pull upstream master git push origin master The git pull command simply does a \u0026ldquo;fast-forward\u0026rdquo; merge of the main branch from an earlier commit up to the latest commit of the upstream\u0026rsquo;s main branch.\nIn other words, when you try to merge a commit with another commit that can be reached by following the first commit\u0026rsquo;s history, Git simplifies things and just moves the pointer forward because there is no divergent change to merge. This is known as a \u0026ldquo;fast-forward\u0026rdquo; merge.\nWhen your main development branch falls behind the corresponding upstream branch, the upstream\u0026rsquo;s main development branch continues linearly from the last commit of your main development branch, assuming there are no additional commits on your main development branch.\nThe figure below shows this situation: the local master stopped at C3, while upstream/master went on to C9, and the topic branch, to C6.\nThe local master stayed at C3, while the upstream\u0026rsquo;s master advanced to C9. With such a commit history, when the upstream\u0026rsquo;s main development branch is merged into your main branch, the merge is done simply by moving the pointer of your main branch forward to the last commit of the upstream\u0026rsquo;s main branch.\nAfter the fast-forward merge, master and upstream/master point to C9. After the merge is complete, the main development branches of both the upstream and your repository point to the same commit.\nAmending the last commit # This is an optional step to redo the last commit. Sometimes we realize, after making a commit, that some files needed more changes or that the commit message needs to be changed. In these cases, it is necessary to do an \u0026ldquo;amend\u0026rdquo; (literally, to \u0026ldquo;mend\u0026rdquo; the last commit).\nIn the figure below, the last commit of the topic branch is C6:\nA topic branch with three commits, C4 to C6; the last one, C6, is what the amend will redo. Git lets you take the last commit, modify the changes made in that commit, and reapply the new set of changes as a new commit that overwrites the last one.\nIn the figure below, C6 was replaced by C6\u0026rsquo;:\nAfter the amend, C6 gives way to a new commit, C6'. To do so, first make the necessary changes to the files. Then stage them normally for commit. Then, do the amend as in the commands below. The commit message can be modified when the editor configured for Git appears.\ngit add FILES git commit --amend Although the example above shows only the git add command to modify or add files, amending the last commit can also involve removing, moving, and renaming files with git rm and git mv.\nTo update only the commit message without modifying files, just run git commit --amend.\nRebasing commits # This is an optional step to keep the commit history as linear as possible.\nThe main development branch may have diverged since the topic branch was created:\nThe topic branch left C1, and master has advanced to C3 since then. It may be a good idea to move the commits of the topic branch and place them on top of the main development branch, so that the topic branch extends linearly from the last commit of the main development branch.\nThe result is the one in the figure below:\nAfter the rebase, the topic branch starts from C3, the last commit of master, and its commits are recreated as C4\u0026rsquo;, C5\u0026rsquo; and C6'. The following commands show how to rebase the topic branch onto the main development branch.\ngit checkout TOPIC-BRANCH git rebase master Editing commits # This is an optional step to keep the commit history clean and concise.\nA pull request may contain several commits. Sometimes it may be necessary to amend a commit that is not the last one. In these cases, we do what is called an interactive rebase.\nAfter developing what is needed on the topic branch, the developer or a reviewer may spot problems that must be dealt with before merging into the upstream. This can lead to several new commits on the topic branch that ideally should have been part of the first commit.\nIn the figure below, these are the three commits C4, C5 and C6 of the topic branch:\nThree commits on the topic branch, C4 to C6, that would fit in one. In these cases, it may be a good idea to do what is called a squash (to crush) of several commits into a single coherent commit with all the related changes.\nAfter the squash, the commits of the topic branch become one, C4\u0026rsquo;, here already on top of C3. The following example shows how to amend the last 3 commits:\ngit checkout TOPIC-BRANCH git rebase -i HEAD~3 Running these commands will open an editor with three lines, one for each of the last 3 commits, ordered from the most recent to the oldest, with instructions on how to edit each line. Something similar to the following:\npick f7f3f6d changed my name a bit pick 310154e updated README formatting and added blame pick a5f4a0d added cat-file # Rebase 710f0f8..a5f4a0d onto 710f0f8 # # Commands: # p, pick = use commit # r, reword = use commit, but edit the commit message # e, edit = use commit, but stop for amending # s, squash = use commit, but meld into previous commit # f, fixup = like \u0026quot;squash\u0026quot;, but discard this commit's log message # x, exec = run command (the rest of the line) using shell # # These lines can be re-ordered; they are executed from top to bottom. # # If you remove a line here THAT COMMIT WILL BE LOST. # # However, if you remove everything, the rebase will be aborted. # # Note that empty commits are commented out For example, to squash all three commits into one, leave the first commit unchanged, replace pick with squash on the next two lines, save, and exit the editor. The editor opens again; clear the commit message, save, and exit the editor.\nBesides pick and squash, there are other operations available that are beyond the scope of this article. But just follow the instructions in the editor to use them. Git will guide you through operations that require several steps, offering suggestions and commands for each step.\nForce push # The steps in the last sections overwrite the branch history. If these steps are performed after the branch was pushed for a pull request on GitHub, it is necessary to use the -f or --force option when pushing:\ngit push -f origin TOPIC-BRANCH The option is only necessary if you are pushing to an already existing pull request branch and when overwriting the branch history. Keep in mind that rewriting history is generally not a good idea, and this is one of the rare scenarios where such an action is safe, given that the commits are being applied to a personal branch of a personal fork, not affecting the upstream repository.\nDeleting a branch # Once an upstream developer merges your pull request, you can delete the topic branch from your local repository and from your fork.\ngit checkout master git branch -D TOPIC-BRANCH git push -d origin TOPIC-BRANCH The next section, Merging a Pull Request, explains how an upstream owner can merge pull requests from contributors.\nMerging a Pull Request # This section is for developers who own an upstream repository and who handle pull requests from contributors.\nThere are two more common methods for merging commits: one that does not introduce an additional commit for the merge; and another that does. Both are perfectly acceptable and will be discussed next.\nWhich method to choose depends on whether you want to keep a concise commit history consisting only of development commits or whether you want to introduce additional commits for each merge in your commit history.\nMerge without a merge commit # Clone the upstream repository to your local system.\ngit clone https://GITHUB/UPSTREAM-OWNER/REPO.git cd REPO If the repository was already cloned earlier, make sure the local development branch is up to date with its counterpart in the upstream repository.\ngit checkout master git pull Create a temporary branch (pr in the example) to pull the contribution (pull request) from a branch of the CONTRIBUTOR\ngit checkout -b pr git pull https://GITHUB/CONTRIBUTOR/REPO.git TOPIC-BRANCH Another way to pull the contribution is to use the number that GitHub associates with each PR. In the following command, $ID refers to that number\ngit fetch origin pull/$ID/head:pr git checkout pr If the main development branch has diverged from the branch in the pull request, the command above will create a new merge commit when pulling. This is not a problem and, if it is what you want, you can skip the next three points.\nIf you want to get rid of the added merge commit, follow only one of the following options:\nAsk the contributor to rebase the topic branch onto the main development branch. See the section Rebasing commits for more details. After such a rebase, delete the temporary branch\ngit checkout master git branch -D pr Start the pull of the pull request again. In fact, if at any moment the contributor rebases, edits, squashes, amends, or drops commits in the pull request, delete the temporary branch and start over, pulling the pull request.\nGo to the pull requests page on GitHub, click the menu next to the Merge pull request button, select Rebase and merge, and click Rebase and merge.\nCaveat: the rebase operation rewrites the commit history of the pull request submitted by the contributor. As a result, it modifies the commit hash, the committer, and the commit date of each change in the pull request. These fields are distinct from the author and author date fields, as you can see with git log --format=fuller. So, usually this is not a big problem.\nRebase the pull request onto the main development branch as explained in the section Rebasing commits\nCaveat: besides those already mentioned in the previous point, since in this case the rebase operation is local, GitHub does not track the changes in the hashes and does not close the pull request automatically after the merge is pushed to the upstream repository. Closing can be done manually, but it will not appear as merged but as closed, which can be a bit confusing for the other collaborators.\nAfter testing, merge the commits in the pull request into the main development branch and remove the pull request branch.\ngit checkout master git merge pr git branch -d pr Finally, push the current state of the development branch to the upstream repository.\ngit push origin master The git merge command above does a fast-forward of the main branch from an earlier commit up to the last commit of the pull request, with no merge commit being created. It works this way because, after the rebase, the pull request branch becomes a linear extension of the main development branch, as the figure below shows:\nAfter the rebase, the pull request branch extends master in a straight line. So, the merge is done simply by advancing the development branch pointer to the last commit of the pull request branch.\nAfter the fast-forward merge, master points to C6\u0026rsquo;, the same commit as pr. Merge with a merge commit # Clone the upstream repository to your local system.\ngit clone https://GITHUB/UPSTREAM-OWNER/REPO.git cd REPO If the repository has already been cloned earlier, make sure the main development branch is up to date with that of the upstream repository\ngit checkout master git pull Create a temporary branch (pr in the example) to pull the contribution (pull request) from the contributor\u0026rsquo;s branch\ngit checkout -b pr git pull https://GITHUB/CONTRIBUTOR/REPO.git TOPIC-BRANCH Another way to pull the contribution is to use the number that GitHub associates with each PR. In the following command, $ID refers to that number\ngit fetch origin pull/$ID/head:pr git checkout pr If the contributor later adds new commits to the pull request, run the commands again to pull the new commits.\nAfter testing, merge the commits in the pull request into the main development branch.\ngit checkout master git merge --no-ff pr git branch -d pr Finally, push the current state of the main development branch (with the commits of the pull request) to the upstream repository.\ngit push origin master The --no-ff (no fast-forward) option of the git merge command ensures that a merge commit is created even if a fast-forward merge is possible.\nThe figure below shows the result: a merge commit, C7, with two parents.\nWith \u0026ndash;no-ff, the merge creates commit C7, which has two parents: C3 and C6'. Conclusion, references, and usage examples # It was a long article, but I believe it is a good reference to consult while you get used to working with Git. Over time, collaborating on projects, many of these commands will become natural to you, as will the dynamics of branches, merges, and rebases. This article is for continuous consultation and support, not to be memorized.\nSome parts of this article were adapted from this repository by the user Susam on GitHub, as well as from the Pro Git book, available on the Git website, respecting the licenses of each reference.\nIf you want to see an example of using Git in a real project, see this playlist from the channel (in Portuguese) where a small Python project is built following the Git development workflow presented here.\nSee you next time!\n","date":"2021-09-20","externalUrl":null,"permalink":"/2021/09/definitive-guide-git-github/","section":"Blog","summary":"Git can be a somewhat scary tool for beginners because of its large number of commands and possibilities. In this article, we demystify the main aspects of Git so that you can collaborate on any project. The famous workflow of forks and pull requests, popularized by GitHub, is covered.","title":"Git and GitHub: The Definitive Collaboration Guide","type":"posts"},{"content":"","date":"2021-09-16","externalUrl":null,"permalink":"/tags/generators/","section":"tags","summary":"","title":"generators","type":"tags"},{"content":"","date":"2021-09-16","externalUrl":null,"permalink":"/tags/geradores/","section":"tags","summary":"","title":"geradores","type":"tags"},{"content":"","date":"2021-09-16","externalUrl":null,"permalink":"/tags/yield/","section":"tags","summary":"","title":"yield","type":"tags"},{"content":"Have you ever seen the term yield from in some Python code and wondered what it was? In this article we will dive even deeper into generators and understand, with examples, what yield from means and how we can use it to make our code even more efficient.\nOrigin # In the article about generators we saw that they originated in the context of coroutines, but that they are applicable in many other contexts, so much so that you don\u0026rsquo;t even need to know about coroutines to use them. yield from was made official in PEP 380, still in a very technical way, aimed at that specific context. But let\u0026rsquo;s take some excerpts from that PEP and bring them closer to simple examples.\nThe abstract of PEP 380 reads:\nA syntax is proposed for a generator to delegate part of its operations to another generator. This allows a section of code containing \u0026lsquo;yield\u0026rsquo; to be factored out and placed in another generator. Additionally, the subgenerator is allowed to return with a value, and the value is made available to the delegating generator. The new syntax also opens up some opportunities for optimisation when one generator re-yields values produced by another.\nThe key point is in the first sentence: a generator delegating part of its operations to another generator.\nAnother quote comes from the language\u0026rsquo;s documentation on expressions, where we have:\nWhen yield from \u0026lt;expr\u0026gt; is used, it treats the supplied expression as a subiterator. (\u0026hellip;) the supplied expression must be an iterable.\nThat is, we can convert an iterable into an iterator and yield from it. Interesting. And, in case you need to refresh these concepts, see this article about iterators and iterables with many examples.\nLet\u0026rsquo;s see how to apply what was presented in a few simple examples.\nSimple examples # I recently wrote about infinite sequences and in that article we saw the islice function. Let\u0026rsquo;s use all the knowledge acquired there.\nConsuming from two generators # Consider that we have two generators of infinite sequences, one for positive even integers and another for positive odd integers:\nfrom itertools import islice def positive_evens(): value = 0 while True: yield value value += 2 def positive_odds(): value = 1 while True: yield value value += 2 Now suppose we want a generator of positive integers, without distinguishing even from odd. We can reuse the generators we already created:\ndef positive_integers(): for even, odd in zip(positive_evens(), positive_odds()): yield even yield odd Let\u0026rsquo;s check the first 10 values generated:\ngen_int = positive_integers() tuple(islice(gen_int, 10)) (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) It worked\u0026hellip; but how? Let\u0026rsquo;s start by looking at the documentation of the zip function:\nhelp(zip) Help on class zip in module builtins: class zip(object) | zip(*iterables) --\u0026gt; A zip object yielding tuples until an input is exhausted. | | \u0026gt;\u0026gt;\u0026gt; list(zip('abcdefg', range(3), range(4))) | [('a', 0, 0), ('b', 1, 1), ('c', 2, 2)] | | The zip object yields n-length tuples, where n is the number of iterables | passed as positional arguments to zip(). The i-th element in every tuple | comes from the i-th iterable argument to zip(). This continues until the | shortest argument is exhausted. | | Methods defined here: | | __getattribute__(self, name, /) | Return getattr(self, name). | | __iter__(self, /) | Implement iter(self). | | __next__(self, /) | Implement next(self). | | __reduce__(...) | Return state information for pickling. | | ---------------------------------------------------------------------- | Static methods defined here: | | __new__(*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature. See that zip receives iterables and generates tuples until those iterables are consumed. Like this:\nzip(positive_evens(), positive_odds()) \u0026lt;zip at 0x7f25b857f300\u0026gt; next(zip(positive_evens(), positive_odds())) (0, 1) Let\u0026rsquo;s look at the first 5 pairs generated:\ntuple(islice(zip(positive_evens(), positive_odds()), 5)) ((0, 1), (2, 3), (4, 5), (6, 7), (8, 9)) So, basically, the for loop in the body of the generator goes through each item of each tuple, making it available (yield) to calls to the generator.\nNow, if the zip function generates tuples, and tuples are iterables, we can pass these tuples to yield from according to the documentation seen earlier. So, we can redefine our generator as:\ndef positive_integers(): for tup in zip(positive_evens(), positive_odds()): yield from tup gen_int = positive_integers() tuple(islice(gen_int, 10)) (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) Quite simple and concise.\nConsuming from another generator # Let\u0026rsquo;s now better understand the part of the PEP about delegating part of the operations to a generator. We will create a generator of positive integers divisible by 10. We know that to be divisible by 10, a number must end in 0 and is therefore an even number. So it makes sense to use the already created generator of even numbers and generate only those divisible by 10 on demand:\ndef positive_integers_divisible_by_ten(): even = positive_evens() yield from (p for p in even if (p % 10 == 0)) Note that we created a generator expression and we are generating from it with yield from.\ngen_ten = positive_integers_divisible_by_ten() tuple(islice(gen_ten, 10)) (0, 10, 20, 30, 40, 50, 60, 70, 80, 90) \u0026ldquo;Flattening\u0026rdquo; lists of lists # Consider that you have a list of lists containing numbers and would like to generate these numbers as if they belonged to a single list. This process is called flatten, meaning to flatten the sublists into the main list.\nlist_of_sublists = [[1, 2, 3], [4, 5, 6], [7], [8, 9]] One way to write a generator would be:\ndef flatten(lst): for sublist in lst: for item in sublist: yield item gen = flatten(list_of_sublists) tuple(gen) (1, 2, 3, 4, 5, 6, 7, 8, 9) Now, each sublist is an iterable and, therefore, we can pass it directly to yield from:\ndef flatten(lst): for sublist in lst: yield from sublist gen = flatten(list_of_sublists) tuple(gen) (1, 2, 3, 4, 5, 6, 7, 8, 9) This flattening has many real applications and this approach is not necessarily the most efficient, read more here. But it is a good way to see more applications of yield from.\nConclusions # Did you enjoy learning more about generators? They are a very efficient way to handle many situations where it is not possible, or not healthy from a resource standpoint, to store all the values beforehand.\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by searching for the tag \u0026ldquo;drops\u0026rdquo; here on the site.\nSee you next time!\n","date":"2021-09-16","externalUrl":null,"permalink":"/2021/09/generators-in-python-yield-from/","section":"Blog","summary":"Have you ever seen the term “yield from” in some Python code and wondered what it was? In this article we will dive even deeper into generators and understand, with examples, what “yield from” means and how we can use it to make our code even more efficient.","title":"yield from - What Is It? Understanding Generators in Python","type":"posts"},{"content":"Did you know you can create an infinite sequence without running into memory problems and still consume that sequence however you like? As your understanding of the Python language grows, you start exploring the modules in the Python standard library. Today we will look at a specific function from the itertools module, islice, which is excellent for our purpose.\nGetting to know the function # Whenever you hear about some feature of your favorite programming language (or the language that pays your bills\u0026hellip;) get to know its documentation. The documentation of the itertools module describes it as:\nThis module implements a number of iterator building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python. The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Together, they form an \u0026ldquo;iterator algebra\u0026rdquo; making it possible to construct specialized tools succinctly and efficiently in pure Python. (\u0026hellip;)\nNote that the documentation mentions iterators. So, if you are not very familiar with the concept, I recommend reading this article here on the site, where I explain in detail what it is about. Then come back to this article :-).\nAs for the islice function, the documentation describes it as:\nitertools.islice(iterable, stop) or itertools.islice(iterable, start, stop[, step]). Make an iterator that returns selected elements from the iterable. (\u0026hellip;)\nA very concise description. Let\u0026rsquo;s start by importing the function:\nfrom itertools import islice Creating our infinite sequence # Now let\u0026rsquo;s create our infinite sequence. If you have already read the article about generators you know that creating such sequences is perfectly possible. Just create a generator and produce the values on demand:\ndef positive_evens(): value = 0 while True: yield value value += 2 Note that the generator will produce non-negative even integers starting from zero whenever requested. Let\u0026rsquo;s quickly recall the behavior of a generator like this. First, seeing that Python recognizes it as a generator:\npositive_evens() \u0026lt;generator object positive_evens at 0x7fac9d1deeb0\u0026gt; Let\u0026rsquo;s bind it to a variable so we can consume it through next calls:\np = positive_evens() next(p) 0 next(p) 2 next(p) 4 And we could go on indefinitely. Now let\u0026rsquo;s understand what islice does.\nConsuming the sequence in a controlled way # Let\u0026rsquo;s take a more detailed look at the description of the function:\nhelp(islice) Help on class islice in module itertools: class islice(builtins.object) | islice(iterable, stop) --\u0026gt; islice object | islice(iterable, start, stop[, step]) --\u0026gt; islice object | | Return an iterator whose next() method returns selected values from an | iterable. If start is specified, will skip all preceding elements; | otherwise, start defaults to zero. Step defaults to one. If | specified as another value, step determines how many values are | skipped between successive calls. Works like a slice() on a list | but returns an iterator. | | Methods defined here: | | __getattribute__(self, name, /) | Return getattr(self, name). | | __iter__(self, /) | Implement iter(self). | | __next__(self, /) | Implement next(self). | | __reduce__(...) | Return state information for pickling. | | __setstate__(...) | Set state information for unpickling. | | ---------------------------------------------------------------------- | Static methods defined here: | | __new__(*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature. Note that there are two signatures. The first is islice(iterable, stop). So let\u0026rsquo;s pass our iterable (the generator) and a stopping value, for example, 10:\nislice(p, 10) \u0026lt;itertools.islice at 0x7fac9c16ee50\u0026gt; Note that it really is an iterator. Let\u0026rsquo;s then consume it by passing it to a tuple:\ntuple(islice(p, 10)) (6, 8, 10, 12, 14, 16, 18, 20, 22, 24) How interesting. We had already started consuming our generator, reaching the number 4. Now the next 10 integers were generated on demand. There was no need for next calls or for building a for loop or anything like that. islice takes care of it. And we can request the next 10 integers the same way:\ntuple(islice(p, 10)) (26, 28, 30, 32, 34, 36, 38, 40, 42, 44) We now have perfect control over the consumption of this sequence.\nThe generator function itself could be passed as an argument, for example:\ntuple(islice(positive_evens(), 20)) (0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38) The first 20 non-negative even integers were generated directly.\nThe famous Fibonacci sequence # Let\u0026rsquo;s face it: the sequence of positive even numbers is not very practical. But the Fibonacci sequence is quite famous and has its applications. Let\u0026rsquo;s use it and learn a bit more about islice. First, let\u0026rsquo;s implement a generator for the Fibonacci sequence:\ndef fibonacci(): a, b = 0, 1 while True: yield a b = a + b yield b a = a + b Let\u0026rsquo;s look at the beginning of the sequence to check our implementation:\nf = fibonacci() f \u0026lt;generator object fibonacci at 0x7fac9c16a6d0\u0026gt; next(f) 0 next(f) 1 next(f) 1 next(f) 2 OK, it works. We can, again, use islice as we did before, looking at the first 10 numbers of the sequence:\ntuple(islice(fibonacci(), 10)) (0, 1, 1, 2, 3, 5, 8, 13, 21, 34) We can also see how to use the second signature of the function: islice(iterable, start, stop[, step]).\nFor example, we can see which numbers lie between positions 5 and 10:\ntuple(islice(fibonacci(), 5, 10)) (5, 8, 13, 21, 34) But be careful with the interpretation. The 5 refers to the index, and indices in Python start at zero. So, note that the values shown actually start from the sixth in the sequence (compare with the result of the previous cell). And the last index is exclusive, similar to a slice of Python lists.\nThe last parameter is the step, also similar to that of lists:\ntuple(islice(fibonacci(), 5, 10, 2)) (5, 13, 34) If you want one specific position, just call next on it and pass None as stop:\nnext(islice(fibonacci(), 5, None)) 5 Conclusion # One more step in understanding and using generators. I hope the articles are connecting well with each other and adding to your knowledge.\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the tag \u0026ldquo;drops\u0026rdquo; here on the site.\nSee you next time!\n","date":"2021-09-09","externalUrl":null,"permalink":"/2021/09/infinite-sequences-python-fibonacci/","section":"Blog","summary":"Did you know you can create an infinite sequence without running into memory problems and still consume that sequence however you like? Today we will look at a specific function from the itertools module, islice, which is excellent for our purpose. We will learn to use it with the example of the Fibonacci sequence.","title":"Infinite Sequences in Python - Fibonacci Like You Have Never Seen","type":"posts"},{"content":"","date":"2021-09-09","externalUrl":null,"permalink":"/tags/itertools/","section":"tags","summary":"","title":"itertools","type":"tags"},{"content":"","date":"2021-09-02","externalUrl":null,"permalink":"/tags/all/","section":"tags","summary":"","title":"all","type":"tags"},{"content":"","date":"2021-09-02","externalUrl":null,"permalink":"/tags/any/","section":"tags","summary":"","title":"any","type":"tags"},{"content":"I have already written here on the site about short-circuiting in the or and and operators in Python. In this article, we will verify that the concept of short-circuiting also applies to the all and any functions, available in a standard installation of the language. We will also see what each of these functions is useful for and what to watch out for when using them, based on a complexity analysis.\nall # Let\u0026rsquo;s start by checking what the documentation says about the function:\nhelp(all) Help on built-in function all in module builtins: all(iterable, /) Return True if bool(x) is True for all values x in the iterable. If the iterable is empty, return True. As the documentation indicates, if every value x of an iterable passed to the function returns True in bool(x), the function returns True. An iterable, according to the documentation, is nothing more than an object that can return its members one at a time, such as sequence types like lists, strings, and tuples. Using a list as an example, we can verify that it is an iterable through a simple for loop:\nexample_list = [1, 2, 3, 4] for number in example_list: print(number) 1 2 3 4 What for does is pass the iterable, in the example the list, to the iter() function, which returns an iterator, from which for requests each item successively through a next() call until the iterable is completely consumed. In the example, the list is consumed by passing each item successively to the print function.\nIf you are still in doubt, I strongly recommend reading this article about iterators and iterables, where the concept is explained in detail with several examples.\nLet\u0026rsquo;s go to a simple example, passing some sequences to the all function:\nall_true = (True, True, True) all(all_true) True If at least one value is False, the function returns False:\nsome_true = (True, False, True) all(some_true) False We have already seen in other articles that the interpreter considers some cases as False. Here are some examples:\nprint(bool(False)) # the False keyword itself print(bool(None)) # the None keyword print(bool(0)) # the integer 0 (zero) print(bool(\u0026#34;\u0026#34;)) # an empty string print(bool(())) # an empty tuple print(bool([])) # an empty list print(bool({})) # an empty dictionary False False False False False False False So we can build a more complete example in which the return of the all function is True:\nall_true = (True, \u0026#34;Chico\u0026#34;, 1, (1, 2), [1, 2], {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2}) all(all_true) True The following example returns False because there is an empty string among the values passed:\nsome_true = (True, \u0026#34;\u0026#34;, 1, (1, 2), [1, 2], {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2}) all(some_true) False The next example returns False because there is an integer with value zero among the values passed:\nsome_true = (True, \u0026#34;Chico\u0026#34;, 0, (1, 2), [1, 2], {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2}) all(some_true) False I hope these examples were enough to understand the function\u0026rsquo;s return. In the last two, since there is at least one element that returns False when passed as an argument to bool(), the return of all() will be False. Now, is it necessary to analyze every value passed in order to get the return? To evaluate this, let\u0026rsquo;s create the following function:\ndef executed_function(value): print(\u0026#34;I was executed!\u0026#34;) return value Note that it is a very simple function: it receives a value, which can be of any type, prints the string \u0026ldquo;I was executed!\u0026rdquo;, and returns the same value passed. Let\u0026rsquo;s do a small test, passing the string \u0026ldquo;hi\u0026rdquo; as the function\u0026rsquo;s argument:\nexecuted_function(\u0026#39;hi\u0026#39;) I was executed! 'hi' Note the message \u0026ldquo;I was executed!\u0026rdquo; and the return of the string passed to the function. But what is the purpose of this function?\nConsider the code cell below. In it there is a list called test_list. Note that all of its items return True when passed as an argument to the bool function, except the third item, because it is the integer zero. This means that passing such a list to the all function should return False. But does the all function need to evaluate all the values?\nTo find out, let\u0026rsquo;s create a generator using a generator expression. Such an expression produces a generator that, instead of returning all the values at once, returns them on demand. For more details, see this article on the site about generators. See in the cells below how the generator_expression returns each value on demand from the next function:\ntest_list = [True, \u0026#34;Chico\u0026#34;, 0, (1, 2), [1, 2], {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2}] generator_expression = (executed_function(item) for item in test_list) generator_expression \u0026lt;generator object \u0026lt;genexpr\u0026gt; at 0x7f2b1d5eec10\u0026gt; next(generator_expression) I was executed! True next(generator_expression) I was executed! 'Chico' next(generator_expression) I was executed! 0 I think you get the idea: we could keep requesting each item until the list is fully consumed. If you want all the remaining values at once, just pass the generator to a container such as, for example, a list:\nlist(generator_expression) I was executed! I was executed! I was executed! [(1, 2), [1, 2], {'a': 1, 'b': 2}] Now the generator is exhausted, so a new next will raise a StopIteration exception:\nnext(generator_expression) --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) \u0026lt;ipython-input-16-7ac23c2b0c8d\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(generator_expression) StopIteration: Note that the behavior of a generator is that of an iterable, according to the concept at the beginning of the article. Since the generator was exhausted, let\u0026rsquo;s create it again and pass it to the all function:\ngenerator_expression = (executed_function(item) for item in test_list) all(generator_expression) I was executed! I was executed! I was executed! False Note that the message \u0026ldquo;I was executed!\u0026rdquo; was displayed three times. That makes sense, because the first two items return True when passed as an argument to the bool function, while the third, the integer zero, returns False. Since the purpose of the all function is to return True only if all items return True, it is not necessary to evaluate the remaining items after finding a False case. Thus, we have shown that the all function exhibits short-circuit behavior.\nOK, but what do we do with this information? What is the practical consequence? Let\u0026rsquo;s see what happens if we pass a large iterable, in this case a list of one million values:\ngiant_list_all_true = [True] * 1_000_000 All the values in this list are True, which means the return of the all function should be True. But, from what we saw earlier, the function goes through every value before giving its return. We saw in this article that IPython and Jupyter Notebook have their own functions (\u0026ldquo;magics\u0026rdquo; in the documentation) that allow some interesting analyses. One of them is %timeit, which measures how long a given command takes. Since this article is being written in a Jupyter Notebook, we can use this function. Let\u0026rsquo;s see the time when passing this list with one million True values:\n%timeit all(giant_list_all_true) 3.72 ms ± 74.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Let\u0026rsquo;s compare this figure with the time for lists that have at least one False value. We already know the return will be False, but how long will it take? Let\u0026rsquo;s start by generating a list, also with one million items, with a False value right at the beginning, at index 10:\ngiant_list_with_false_index_10 = giant_list_all_true.copy() giant_list_with_false_index_10[10] = False %timeit all(giant_list_with_false_index_10) 98.1 ns ± 0.44 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Note the huge difference! We went from milliseconds (10-3 seconds) to nanoseconds (10-9 seconds). A million-fold improvement! But that is because the value was right at the beginning. Let\u0026rsquo;s create another list of one million items, with a single False but in the last position:\ngiant_list_with_false_last_index = giant_list_all_true.copy() giant_list_with_false_last_index[-1] = False %timeit all(giant_list_with_false_last_index) 3.8 ms ± 123 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Note that we are back in the millisecond range, a value almost equal to that of the list with all True values. That makes sense: each value was analyzed sequentially until the last one was reached.\nJust to close the reasoning, let\u0026rsquo;s put a single False exactly in the middle of the list:\ngiant_list_with_false_middle_index = giant_list_all_true.copy() giant_list_with_false_middle_index[499_999] = False %timeit all(giant_list_with_false_middle_index) 1.83 ms ± 3.31 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) As expected, we got a value that is practically half of the previous one.\nThat is, we have a function with linear behavior with respect to a time complexity analysis. In the worst case, it will have to evaluate all the items to be able to return. Keep this in mind if you intend to use this function on a large amount of data, especially if there is reason to believe there will be few False values, since if they are at the end of the iterable, a long time will be needed. If you want to see the C source code of this function and check the implementation, click here.\nany # Let\u0026rsquo;s follow the same script we used earlier for the all function, starting with the documentation:\nhelp(any) Help on built-in function any in module builtins: any(iterable, /) Return True if bool(x) is True for any x in the iterable. If the iterable is empty, return False. Again, this function receives an iterable. But, in this case, it returns True if at least one of the items returns True when passed as an argument to the bool function. Let\u0026rsquo;s see two examples. In the first, we pass only False values:\nall_false = (False, False, False) any(all_false) False As expected, we got False as the return. If at least one of the values passed is True, that is enough for us to get True as the return:\nsome_false = (False, True, False) any(some_false) True As we have already seen, we do not necessarily need to pass True or False directly: we can pass other types that will be True or False when passed to the bool function. Let\u0026rsquo;s redefine our test_list variable below. Note that the first three items return False when passed to the bool function (see the examples of values considered False above if you have trouble understanding). Let\u0026rsquo;s pass this list to our generator expression:\ntest_list = [False, \u0026#34;\u0026#34;, 0, (1, 2), [1, 2], {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 2}] generator_expression = (executed_function(item) for item in test_list) generator_expression \u0026lt;generator object \u0026lt;genexpr\u0026gt; at 0x7f2b1d5eef20\u0026gt; And pass this expression to the any function:\nany(generator_expression) I was executed! I was executed! I was executed! I was executed! True Note that the function was executed 4 times. Upon finding the first item that returns True, it is no longer necessary to keep consuming the generator, because a single True is enough for the function to return True. Again, short-circuit behavior.\nSo, as we did for all, let\u0026rsquo;s see what happens if we pass a large iterable, in this case a list of one million values:\ngiant_list_all_false = [False] * 1_000_000 All the values in this list are False, which means the return of the any function should be False. But, from what we saw earlier, the function goes through every value before giving its return. Let\u0026rsquo;s see the time when passing this list with one million False values:\n%timeit any(giant_list_all_false) 3.18 ms ± 93.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Let\u0026rsquo;s compare this figure with the time for lists that have at least one True value. We already know the return will be True, but how long will it take? Let\u0026rsquo;s start by generating a list, also with one million items, with a True value right at the beginning, at index 10:\ngiant_list_with_true_index_10 = giant_list_all_false.copy() giant_list_with_true_index_10[10] = True %timeit any(giant_list_with_true_index_10) 94.4 ns ± 0.355 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Note the huge difference! We went from milliseconds (10-3 seconds) to nanoseconds (10-9 seconds). A million-fold improvement! But that is because the value was right at the beginning. Let\u0026rsquo;s create another list of one million items, with a single True but in the last position:\ngiant_list_with_true_last_index = giant_list_all_false.copy() giant_list_with_true_last_index[-1] = True %timeit any(giant_list_with_true_last_index) 3.18 ms ± 41.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Note that we are back in the millisecond range, a value almost equal to that of the list with all False values. That makes sense: each value was analyzed sequentially until the last one was reached.\nJust to close the reasoning, let\u0026rsquo;s put a single True exactly in the middle of the list:\ngiant_list_with_true_middle_index = giant_list_all_false.copy() giant_list_with_true_middle_index[499_999] = True %timeit any(giant_list_with_true_middle_index) 1.54 ms ± 3.18 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) As expected, we got a value that is practically half of the previous one.\nThat is, just like all, any is also a function with linear behavior with respect to a time complexity analysis. In the worst case, it will have to evaluate all the items to be able to return. Keep this in mind if you intend to use this function on a large amount of data, especially if there is reason to believe there will be few True values, since if they are at the end of the iterable, a long time will be needed. If you want to see the C source code of this function and check the implementation, click here.\nConclusion # That\u0026rsquo;s it for this article. I hope you understood it and learned something new. We went through several concepts, such as iterables, iterators, and generator functions, and did a bit of time complexity analysis of algorithms. A lot of added value, I am sure.\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the tag \u0026ldquo;drops\u0026rdquo; here on the site.\nNote\nThe time reported by %timeit may vary from machine to machine, so you will not necessarily get the same values. However, the difference in orders of magnitude should hold.\nSee you next time!\n","date":"2021-09-02","externalUrl":null,"permalink":"/2021/09/any-and-all-functions-in-python/","section":"Blog","summary":"In this article, we will see how the all and any functions, available in a standard installation of Python, work. We will also see what each of them is useful for and what to watch out for when using them, based on a complexity analysis.","title":"The any and all Functions in Python","type":"posts"},{"content":"Do you know the difference between a \u0026ldquo;normal\u0026rdquo; function and a generator function in Python? What is the difference between the return of a usual function and the yield of a generator? In this article we answer these questions and dig deeper into some aspects of the language.\nIf you prefer to watch a video, click the player below (in Portuguese). The complete article follows the video.\nreturn - Regular functions # Let\u0026rsquo;s start by declaring a simple function that receives a sequence of numbers, squares them, and returns those squares in a list. If you need to review the concept of an iterable, see this article, where we used this same function as an example.\ndef square_numbers(numbers_iterable): result = [] for number in numbers_iterable: result.append(number**2) return result Now let\u0026rsquo;s create a tuple of numbers and pass it to that function:\nnumbers = (1, 2, 3) square_numbers(numbers) [1, 4, 9] OK, the behavior we expected. We can check the type of square_numbers and of square_numbers(numbers):\ntype(square_numbers) function type(square_numbers(numbers)) list Notice that the object square_numbers is of type function, as expected. And when the argument numbers was passed, the object became a list. After all, the return of the function is a list. The return statement ends the function, delivering what was requested, the list.\nyield - Generator functions # Let\u0026rsquo;s start by seeing how the language\u0026rsquo;s own documentation defines generators:\nA function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function. Usually refers to a generator function, but may refer to a generator iterator in some contexts. In cases where the intended meaning isn\u0026rsquo;t clear, using the full terms avoids ambiguity.\nI believe that was not very enlightening, but let\u0026rsquo;s fix that. First, to better understand the idea of an iterator, read this article where I explain it in detail. It is important for what we will see here. The documentation also defines the term \u0026ldquo;generator iterator\u0026rdquo;:\nAn object created by a generator function. Each yield temporarily suspends processing, remembering the location execution state (including local variables and pending try-statements). When the generator iterator resumes, it picks up where it left off (in contrast to functions which start fresh on every invocation).\nIn this last definition there are important aspects, especially the idea of temporary suspension. Let\u0026rsquo;s create our generator by modifying our function. Notice that we just removed the creation of the list and replaced the return with yield number**2:\ndef squares_generator(numbers_iterable): for number in numbers_iterable: yield number**2 Before interacting with this generator, let\u0026rsquo;s look at the word yield. Dictionaries give, among others, these meanings:\nto produce to provide to give a return, as an investment does Combining these meanings with the definitions, we can think of a generator as a construct that produces values on demand at each call of next. We will now see that idea in action. Let\u0026rsquo;s start by evaluating the types:\ntype(squares_generator) function squares_generator(numbers) \u0026lt;generator object squares_generator at 0x7f4e15114970\u0026gt; type(squares_generator(numbers)) generator g = squares_generator(numbers) g \u0026lt;generator object squares_generator at 0x7f4e15114a50\u0026gt; Notice that the object squares_generator itself is of type function. However, when we pass the argument, it becomes of type generator, and that object has a reference in memory. Observe that no value was returned. According to the documentation, the values will be generated one at a time on demand:\nnext(g) 1 next(g) 4 next(g) 9 This is really the same behavior we saw in the article about iterators. So, we should expect a StopIteration when the whole iterable passed has been consumed:\nnext(g) --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) \u0026lt;ipython-input-13-e734f8aca5ac\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(g) StopIteration: Generator expressions # The same generator function could be written in a single line using a generator expression:\n(number**2 for number in numbers) \u0026lt;generator object \u0026lt;genexpr\u0026gt; at 0x7f4e14017ba0\u0026gt; Note that its syntax is similar to a list comprehension, but with parentheses instead of brackets. It has the same behavior as the generator created earlier:\nfor square in (number**2 for number in numbers): print(square) 1 4 9 g = (number**2 for number in numbers) next(g) 1 next(g) 4 next(g) 9 next(g) --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) \u0026lt;ipython-input-20-e734f8aca5ac\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(g) StopIteration: Practical applications of generators # More important than knowing that generators exist is knowing how to apply them in real cases. PEP 255, which implements generators in the Python language, presents the rationale that led to the implementation, but it can be somewhat abstract for beginners because it deals with callbacks and coroutines. Here I will try to present uses that are more accessible, but no less important, than the ones presented in the PEP.\nLet\u0026rsquo;s start by creating a list of names and a list of majors that stand for students at some institution:\nnames = [\u0026#39;Americium\u0026#39;, \u0026#39;Californium\u0026#39;, \u0026#39;Copernicium\u0026#39;, \u0026#39;Dysprosium\u0026#39;, \u0026#39;Einsteinium\u0026#39;] courses = [\u0026#39;Geography\u0026#39;, \u0026#39;Film\u0026#39;, \u0026#39;Astronomy\u0026#39;, \u0026#39;Languages - Greek\u0026#39;, \u0026#39;Physics\u0026#39;] Imagine that you want to create a registry, unifying in a single data structure the students\u0026rsquo; names and their majors, besides associating a sequential enrollment number with each student.\nIn the next cell we make two implementations. One is a traditional function, storing each student as a dictionary inside a list and returning that list. The other generates each student dictionary, that is, without storing it in any structure. If storage is desired, it must be done by whoever consumes the generator:\ndef student_registry(names, courses): registry = [] for i, (name, course) in enumerate(zip(names, courses)): student = { \u0026#39;id\u0026#39;: i, \u0026#39;name\u0026#39;: name, \u0026#39;course\u0026#39;: course, } registry.append(student) return registry def student_generator(names, courses): for i, (name, course) in enumerate(zip(names, courses)): student = { \u0026#39;id\u0026#39;: i, \u0026#39;name\u0026#39;: name, \u0026#39;course\u0026#39;: course, } yield student Let\u0026rsquo;s start with the function, seeing the result of passing the variables names and courses as arguments and seeing how long this processing takes. We have already seen in this article that IPython and the Jupyter Notebook have their own functions (\u0026ldquo;magics\u0026rdquo; in the documentation) that allow some interesting analyses. One of them is %timeit, which measures how long a given command takes. Since this article is being written in a Jupyter Notebook, we can use this function.\nstudent_registry(names, courses) [{'id': 0, 'name': 'Americium', 'course': 'Geography'}, {'id': 1, 'name': 'Californium', 'course': 'Film'}, {'id': 2, 'name': 'Copernicium', 'course': 'Astronomy'}, {'id': 3, 'name': 'Dysprosium', 'course': 'Languages - Greek'}, {'id': 4, 'name': 'Einsteinium', 'course': 'Physics'}] %timeit student_registry(names, courses) 1.02 µs ± 21.2 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) Observe that it is a short time, on the order of 1 microsecond. Let\u0026rsquo;s now see the generator. First, let\u0026rsquo;s check that the generator returns no value at all, only an object in memory:\nstudent_generator(names, courses) \u0026lt;generator object student_generator at 0x7f4e15114e40\u0026gt; And that, if we want all the results at once, we need to consume the generator by putting it, for example, in a list:\nlist(student_generator(names, courses)) [{'id': 0, 'name': 'Americium', 'course': 'Geography'}, {'id': 1, 'name': 'Californium', 'course': 'Film'}, {'id': 2, 'name': 'Copernicium', 'course': 'Astronomy'}, {'id': 3, 'name': 'Dysprosium', 'course': 'Languages - Greek'}, {'id': 4, 'name': 'Einsteinium', 'course': 'Physics'}] But if the generator only creates a reference in memory, without actually creating a data structure, how long does that take?\n%timeit student_generator(names, courses) 161 ns ± 1.35 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Note that we dropped to the order of nanoseconds! After all, only a reference was created and each student record will be created on demand:\n%timeit next(student_generator(names, courses)) 534 ns ± 31.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) If only a reference was created and not a data structure, then this should also be reflected in the space occupied, and this is where it gets even more interesting. To analyze the space occupied by each object, we will use the pympler library:\nfrom pympler.asizeof import asizeof asizeof(student_registry(names, courses)) 2400 asizeof(student_generator(names, courses)) 1416 The results are in bytes. Since the function returns a list with the records, in the second-to-last cell we are seeing the space occupied by that list, while in the last cell we see only the space of the generated reference.\nIn case that was not clear, let\u0026rsquo;s do an example that demands more resources. Modifying the logic of the previous function and generator a little, we can make them receive an integer that represents how many students we want to store or generate, respectively. And these students, in our example, will be generated randomly from the two lists created earlier.\nimport random def random_student_registry(count): registry = [] for i in range(count): student = { \u0026#39;id\u0026#39;: i, \u0026#39;name\u0026#39;: random.choice(names), \u0026#39;course\u0026#39;: random.choice(courses), } registry.append(student) return registry def random_student_generator(count): for i in range(count): student = { \u0026#39;id\u0026#39;: i, \u0026#39;name\u0026#39;: random.choice(names), \u0026#39;course\u0026#39;: random.choice(courses), } yield student Let\u0026rsquo;s see an example with 5 random students:\nrandom_student_registry(5) [{'id': 0, 'name': 'Americium', 'course': 'Film'}, {'id': 1, 'name': 'Einsteinium', 'course': 'Languages - Greek'}, {'id': 2, 'name': 'Americium', 'course': 'Languages - Greek'}, {'id': 3, 'name': 'Copernicium', 'course': 'Astronomy'}, {'id': 4, 'name': 'Einsteinium', 'course': 'Languages - Greek'}] random_student_generator(5) \u0026lt;generator object random_student_generator at 0x7f4dfeade430\u0026gt; list(random_student_generator(5)) [{'id': 0, 'name': 'Einsteinium', 'course': 'Geography'}, {'id': 1, 'name': 'Americium', 'course': 'Astronomy'}, {'id': 2, 'name': 'Californium', 'course': 'Languages - Greek'}, {'id': 3, 'name': 'Dysprosium', 'course': 'Geography'}, {'id': 4, 'name': 'Einsteinium', 'course': 'Astronomy'}] Notice that the results are different and there may be repetition, since they are now random.\nBut what happens if we ask for, say\u0026hellip; 1 million students?\n%timeit random_student_registry(1_000_000) 880 ms ± 7.31 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit random_student_generator(1_000_000) 144 ns ± 4.69 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Notice that the function went from the microsecond range to the millisecond range. After all, it now has to build the whole list of records. However, the generator remains in the nanosecond range, because nothing was actually created or stored, only the generator object was stored in memory, and that takes little time.\nAnd what about the space occupied by each object?\nasizeof(random_student_registry(1_000_000)) 272698416 It is even hard to grasp this number in bytes, so let\u0026rsquo;s convert it to megabytes:\n_ / 2**20 # conversion to megabytes 260.0654754638672 A list with 1 million entries, each one a dictionary, occupying 260 MB. Did you imagine that number?\nLet\u0026rsquo;s see the generator:\nasizeof(random_student_generator(1_000_000)) 440 Notice that this is a negligible amount of space. After all, nothing was actually stored except the generator object. Now, if we actually consume this generator, storing each result in, for example, a list, we will get the same values as before:\n%timeit list(random_student_generator(1_000_000)) 854 ms ± 21.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) asizeof(list(random_student_generator(1_000_000))) / 2**20 259.63890075683594 Now, we do not necessarily need to consume the whole generator at once. That is the beauty of generators: we can consume on demand and, therefore, manage computational resources better, both execution time and storage.\nHere on the site I will show uses in the future and link them here, but you can already imagine some: consuming a large database in small pieces; reading large files in parts; the possibility of creating virtually infinite series of values; and so on.\nConclusion # That is it for this article. I hope you understood it and learned something new. We went through several concepts, such as iterables, iterators, functions, generators, and did a bit of analysis of time and space occupied. A lot of added value, I am sure.\nDid you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the \u0026quot;drops\u0026quot; tag here on the site.\nNote\nThe time reported by %timeit may vary from machine to machine, so you will not necessarily get the same values. However, the difference in orders of magnitude should hold.\nSee you next time!\n","date":"2021-08-30","externalUrl":null,"permalink":"/2021/08/python-generators-fast-code/","section":"Blog","summary":"Do you know the difference between a “normal” function and a generator function in Python? What is the difference between the return of a usual function and the yield of a generator? In this article we answer these questions and dig deeper into some aspects of the language. We will see how generators in Python can make your code up to 1000 times faster.","title":"Generators in Python - Code Up to 1000 Times Faster","type":"posts"},{"content":"","date":"2021-08-23","externalUrl":null,"permalink":"/tags/iterable/","section":"tags","summary":"","title":"iterable","type":"tags"},{"content":"","date":"2021-08-23","externalUrl":null,"permalink":"/tags/iterador/","section":"tags","summary":"","title":"iterador","type":"tags"},{"content":"","date":"2021-08-23","externalUrl":null,"permalink":"/tags/iterator/","section":"tags","summary":"","title":"iterator","type":"tags"},{"content":"Do you know what an iterable is? And an iterator? How do you recognize these structures in Python? Answering these questions is the goal of this article.\nIf you prefer to watch a video, click the player below (in Portuguese). The complete article follows the video.\nIterables # Let\u0026rsquo;s start by creating a function that receives a collection of values, squares them, and returns a list with those squares:\ndef square_all(iterable_of_numbers): result = [] for number in iterable_of_numbers: result.append(number**2) return result Note that the function parameter was named iterable_of_numbers. The goal is to show what can be iterable and how to recognize an iterable.\nLet\u0026rsquo;s start with the language documentation. In its glossary, an iterable is defined as:\nAn object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or with a __getitem__() method that implements Sequence semantics.\nIterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), \u0026hellip;). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop.\nNothing beats an example to understand these technicalities. Let\u0026rsquo;s start by creating a variable numbers that will be a tuple of integers, since the glossary says that sequence types are naturally iterable:\nnumbers = (1, 2, 3) Note that the documentation says an iterable can be recognized by an __iter__() or __getitem__() method. To check the attributes and methods available for an object, there is the function dir:\ndir(numbers) ['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index'] Note that these methods are present. If you have trouble finding them in the list, the following code confirms their presence (and also makes you curious to research set in Python \u0026#x1f604; )\nset((\u0026#39;__iter__\u0026#39;, \u0026#39;__getitem__\u0026#39;)) \u0026amp; set(dir(numbers)) {'__getitem__', '__iter__'} OK, numbers is an iterable. Following the glossary text, it is possible to go through each item using a for loop. Look again at the body of the function created at the start of the article and notice that this is exactly what happens.\nSo, it is possible to pass numbers as an argument to our function:\nsquare_all(numbers) [1, 4, 9] As an exercise, make numbers a list and see that it works the same. Create your own exercises with the types mentioned in the glossary to understand the subject even better.\nAnother way to recognize numbers as an iterable is through a list comprehension, since a for loop also occurs in it:\n[number**2 for number in numbers] [1, 4, 9] The glossary also says that an iterable can be passed to language functions such as map. The map function applies a function to all items of an iterable. Let\u0026rsquo;s create an anonymous function that also squares all the items:\nmap(lambda x : x**2, numbers) \u0026lt;map at 0x7fe4d05d5d60\u0026gt; Note that map returns an object in memory that is, in fact, an iterator, our next subject. But first, just to show that it worked, let\u0026rsquo;s pass the map to a list:\nlist(map(lambda x : x**2, numbers)) [1, 4, 9] Iterators # Let\u0026rsquo;s consult the glossary again:\nAn object representing a stream of data. Repeated calls to the iterator\u0026rsquo;s __next__() method (or passing it to the built-in function next()) return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its __next__() method just raise StopIteration again. Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a list) produces a fresh new iterator each time you pass it to the iter() function or use it in a for loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.\nTo understand, let\u0026rsquo;s create an iterator by passing our variable numbers to the iter() function and assigning the result to a new variable num_iter:\nnum_iter = iter(numbers) Let\u0026rsquo;s check the type of the generated variable:\ntype(num_iter) tuple_iterator As the type itself shows, it is an iterator. Being an iterator, following the documentation, there must be an __iter__() method and also a __next__(). Let\u0026rsquo;s check:\ndir(num_iter) ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__length_hint__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__'] set((\u0026#39;__iter__\u0026#39;, \u0026#39;__next__\u0026#39;)) \u0026amp; set(dir(num_iter)) {'__iter__', '__next__'} Now let\u0026rsquo;s understand what the documentation means by successive calls to next():\nnext(num_iter) 1 Note that only the first number was displayed. Let\u0026rsquo;s make three more calls:\nnext(num_iter) 2 next(num_iter) 3 next(num_iter) --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) \u0026lt;ipython-input-16-1d816cd6b265\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(num_iter) StopIteration: As described, when the iterator is exhausted, a StopIteration exception is raised, indicating that the whole iterator has been consumed.\nJust to clear up a question that may be crossing your mind, an iterable is not necessarily an iterator. Example:\nnext(numbers) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) \u0026lt;ipython-input-17-2cf59712f126\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(numbers) TypeError: 'tuple' object is not an iterator See that numbers itself is not an iterator even though it is an iterable.\nLet\u0026rsquo;s now check that the map object generated in the previous section is also an iterator. Recall:\nmap(lambda x : x**2, numbers) \u0026lt;map at 0x7fe4d0507730\u0026gt; Let\u0026rsquo;s assign it to a variable to make the object easier to handle:\nnum_map = map(lambda x : x**2, numbers) Let\u0026rsquo;s see the type of the object and make successive calls to next:\ntype(num_map) map next(num_map) 1 next(num_map) 4 next(num_map) 9 next(num_map) --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) \u0026lt;ipython-input-24-bc498b643d23\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 next(num_map) StopIteration: See that it is the same behavior seen with the object created via iter().\nSince the documentation says that an iterator is also an iterable, we can pass the map object as an argument to our function:\nsquare_all(map(lambda x : x**2, numbers)) [1, 16, 81] Since the anonymous function already squared the values, the for loop inside the square_all function is squaring them again. Hence the values obtained.\nThe for loop handles StopIteration automatically, which is why we don\u0026rsquo;t see the exception when the iterator is consumed.\nWhen the iterator is passed to a container type, it is consumed entirely, also without raising the exception:\nlist(map(lambda x : x**2, numbers)) [1, 4, 9] tuple(map(lambda x : x**2, numbers)) (1, 4, 9) If the iterator has started to be consumed and is then passed to a container, only the rest of the iterator will be included. For example, let\u0026rsquo;s recreate our map iterator and call next once:\nnum_map = map(lambda x : x**2, numbers) next(num_map) 1 Now, let\u0026rsquo;s pass the iterator to a list:\nlist(num_map) [4, 9] Note that only the last two items appear in the list. After all, the iterator keeps no record of past items. That is the meaning of the phrase \u0026ldquo;an object representing a stream of data\u0026rdquo; in the glossary. The iterator had already returned the item 1, the next ones in the stream were 4 and 9, and these were passed to the list when it consumed the iterator entirely.\nIf you want some historical context on iterators in the language, I recommend reading PEP 234.\nConclusion # As you may have noticed, iterators and iterables appear frequently in the language, and you may not even have realized you were using them at various points. Understanding these concepts is crucial for understanding more complex structures of the language such as, for example, generators, which will soon be the topic of an article here on the site.\nDid you like this article? It is part of the Drops of programming, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by searching for the tag \u0026ldquo;drops\u0026rdquo; here on the site.\nSee you next time!\n","date":"2021-08-23","externalUrl":null,"permalink":"/2021/08/iterators-and-iterables-in-python/","section":"Blog","summary":"Do you know what an iterable is? And an iterator? How do you recognize these structures in Python? Answering these questions is the goal of this article.","title":"Iterators and iterables in Python","type":"posts"},{"content":"","date":"2021-08-23","externalUrl":null,"permalink":"/tags/iteravel/","section":"tags","summary":"","title":"iterável","type":"tags"},{"content":"The Python language supports the following operators for booleans:\nOperator Meaning not unary negation and conditional AND or conditional OR The and and or operators are short-circuit operators, meaning that the second operand is only evaluated if the result cannot be determined from the value of the first operand alone. Confusing? Let\u0026rsquo;s look at some examples in this article.\nIf you prefer to watch a video, click the player below. The full article follows the video.\nOverview # and # Suppose we are searching a list data for a given value 'X' and want the index of that value. Python lists have a method for this, but here the goal is to understand the concept, so we will solve it with the most basic tools of the language. At first, let\u0026rsquo;s assume that the value we are looking for is actually in the list:\ndata = [\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;, \u0026#39;D\u0026#39;, \u0026#39;X\u0026#39;, \u0026#39;Z\u0026#39;] A simple way is with a while loop:\nindex = 0 while index \u0026lt; len(data) and data[index] != \u0026#39;X\u0026#39;: index += 1 print(index) 3 Notice in the previous code that we first check whether the index is less than the length of the list and only then check whether the position being examined differs from the desired value. Let\u0026rsquo;s see what happens if we swap the order of the comparison:\nindex = 0 while data[index] != \u0026#39;X\u0026#39; and index \u0026lt; len(data): index += 1 print(index) 3 Notice that, apparently, there is no difference. But what happens if the list does not contain the value we are looking for?\nLet\u0026rsquo;s redefine our list without the value 'X':\ndata = [\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;, \u0026#39;D\u0026#39;, \u0026#39;Z\u0026#39;] Let\u0026rsquo;s run our loop again, checking first whether the index is less than the length of the list:\nindex = 0 while index \u0026lt; len(data) and data[index] != \u0026#39;X\u0026#39;: index += 1 print(index) 4 We got the value 4 for index because the whole list was traversed and the desired value was not found. Thus, all valid indices were visited and, when the first comparison (first operand of and) turned out to be false, the loop was stopped. Notice that index 4 makes no sense for this list, since the largest possible index is 3 (remember that sequences in Python start at index zero).\nNow let\u0026rsquo;s see what happens when we swap the order of the operands:\nindex = 0 while data[index] != \u0026#39;X\u0026#39; and index \u0026lt; len(data): index += 1 print(index) --------------------------------------------------------------------------- IndexError Traceback (most recent call last) \u0026lt;ipython-input-6-d88901ad107a\u0026gt; in \u0026lt;module\u0026gt; 1 index = 0 ----\u0026gt; 2 while data[index] != 'X' and index \u0026lt; len(data): 3 index += 1 4 print(index) IndexError: list index out of range An error occurs because, when index is set to 4, the first operand is data[4] != 'X'. Since index 4 is not valid, we get an IndexError. Notice that Python did not evaluate the second operand, which would have prevented the error. This is the behavior known as short-circuit evaluation.\nThis behavior can be used, as we saw in this example, to avoid possible errors in the evaluation of the second operand. Checking first whether the index is less than the length prevents an IndexError.\nWhat is this behavior based on? On basic Boolean logic. The and operator, the logical conjunction, can only result in True if both operands are True. Thus, a first operand with value False is already enough for the loop to stop. If this is still confusing, take a look at truth tables. I have even written a small Python program to build truth tables, which you can check out here.\nTo complete the article, the truth table for the and operator, generated by the Truth Table Generator project, is the following:\n+--------------+--------------+-----------------------------+ | operand_1 | operand_2 | operand_1 and operand_2 | |--------------+--------------+-----------------------------| | True | True | True | | True | False | False | | False | True | False | | False | False | False | +--------------+--------------+-----------------------------+ Notice that the result is True only when both operands are True. Thus, if the first operand is False, there is no need to evaluate the second. This is the principle behind short-circuit behavior.\nor # Let\u0026rsquo;s look at examples for the or operator. To understand these examples, we need to know that the interpreter treats the following cases as False:\nprint(bool(False)) # the keyword False itself print(bool(None)) # the keyword None print(bool(0)) # the integer 0 (zero) print(bool(\u0026#34;\u0026#34;)) # an empty string print(bool(())) # an empty tuple print(bool([])) # an empty list print(bool({})) # an empty dictionary False False False False False False False If you want a more historical view of this behavior, I recommend reading PEP 285, which implemented the bool type in the Python language.\nLet\u0026rsquo;s then consider the following two situations:\nnickname = \u0026#34;\u0026#34; def executed_function(): return \u0026#34;Function executed\u0026#34; print(nickname or executed_function()) Function executed nickname = \u0026#34;Chico\u0026#34; print(nickname or executed_function()) Chico Again, the behaviors derive from Boolean logic. The or operator, the logical disjunction, can only result in False if both operands are False. Thus, a first operand with value True is already enough for the loop to stop. That is why, in the second case, where nickname is not an empty string, its logical value is True and the function is not executed.\nTo complete the article, the truth table for the or operator, generated by the Truth Table Generator project, is the following:\n+--------------+--------------+----------------------------+ | operand_1 | operand_2 | operand_1 or operand_2 | |--------------+--------------+----------------------------| | True | True | True | | True | False | True | | False | True | True | | False | False | False | +--------------+--------------+----------------------------+ Compare the table with what was written to understand the behavior.\nSummary of the behavior # Operation Result Notes x or y If x is False, then y, else x y is evaluated only if x is False x and y If x is False, then x, else y y is evaluated only if x is True not x If x is True, then False, else True not has lower priority than non-Boolean operators Did you like this article? It is part of the programming Drops, a set of shorter posts focused on fundamentals, covering some aspects of the Python language and of programming in general. You can read more of these articles by browsing the \u0026quot;drops\u0026quot; tag here on the site.\nSee you next time!\n","date":"2021-08-16","externalUrl":null,"permalink":"/2021/08/short-circuit-in-python/","section":"Blog","summary":"The and and or operators are short-circuit operators, meaning that the second operand is only evaluated if the result cannot be determined from the value of the first operand alone. Confusing? Let’s look at some examples in this article.","title":"Short-Circuit Evaluation in Python - Boolean Operators","type":"posts"},{"content":"The Olympics are coming to an end\u0026hellip; and, besides nostalgia and stories of perseverance, every Olympics leaves behind an enormous amount of data! In this article, we will see how to use Pandas to analyze part of that data, looking at how athletes\u0026rsquo; times have evolved in track and field events.\nIf you prefer to watch a video, click the player below. The full article follows the video.\nGetting the data # The first step is to get the data for the analysis. Basically, there are three ways:\ndirect retrieval from a reliable source that makes the data available in a proper form; direct retrieval from a reliable source that does not make the data available in a proper form; indirect retrieval, through third parties. A reliable source would be https://olympics.com/, the website of the International Olympic Committee. However, it does not provide the data in a proper form, such as tables that could be downloaded directly from the site. Nor does it offer an API for requests. So one way to get the data would be scraping it, using, for example, Beautiful Soup or Selenium. Or even Pandas, as I have already shown in this article. But that would require studying the structure of the pages, especially because the site makes use of dynamically generated data.\nNow, think about it\u0026hellip; the Olympics is the most important sporting event in the world, so surely many people have already looked for the data for all sorts of reasons and, eventually, some of them may have made it available in a format suitable for our purposes. And the place where such data would probably be found is Kaggle, a site focused on data science and machine learning.\nTry searching for \u0026ldquo;olympics\u0026rdquo; on Kaggle and you will see a reasonable number of datasets on the subject. In this case, I chose this one, which, according to its creator, contains the track and field data from https://olympics.com/ from 1896 to 2016. Obviously, we will need to test the consistency of the data, since we are getting it indirectly, but that is not much different from what we would do even when getting it directly. Note also that the dataset does not include 2020, which is quite reasonable considering that I am writing during the Olympics. But, as we will see, it is not very hard to insert current data, at least on a small scale.\nWith those considerations made, let\u0026rsquo;s go to the code. We start by importing pandas and checking the data file which, in this case, was saved in a folder called data under the name results.csv. The path to the file was assigned to a variable file:\nimport pandas as pd file = \u0026#39;data/results.csv\u0026#39; Looking at the first 5 and the last 5 lines of the file gives us a general idea of its organization:\n!head -5 data/results.csv Gender,Event,Location,Year,Medal,Name,Nationality,Result M,10000M Men,Rio,2016,G,Mohamed FARAH,USA,25:05.17 M,10000M Men,Rio,2016,S,Paul Kipngetich TANUI,KEN,27:05.64 M,10000M Men,Rio,2016,B,Tamirat TOLA,ETH,27:06.26 M,10000M Men,Beijing,2008,G,Kenenisa BEKELE,ETH,27:01.17 !tail -5 data/results.csv W,Triple Jump Women,Athens,2004,S,Hrysopiyi DEVETZI,GRE,15.25 W,Triple Jump Women,Athens,2004,B,Tatyana LEBEDEVA,RUS,15.14 W,Triple Jump Women,Atlanta,1996,G,Inessa KRAVETS,UKR,15.33 W,Triple Jump Women,Atlanta,1996,S,Inna LASOVSKAYA,RUS,14.98 W,Triple Jump Women,Atlanta,1996,B,Sarka KASPARKOVA,CZE,14.98 The column called Result in the file refers to the athletes\u0026rsquo; times, so I think it makes more sense to name it Time. A visual inspection of the file as a whole shows that some lines also have a column with information about the wind during the competition, even though this column is not named at the beginning of the file. I show this inspection in the video at the start of the article. And why there is wind data will be explained later in the article.\nSo, the following column names seem appropriate to me:\ncolumns = (\u0026#39;Gender\u0026#39;, \u0026#39;Event\u0026#39;, \u0026#39;Location\u0026#39;, \u0026#39;Year\u0026#39;, \u0026#39;Medal\u0026#39;, \u0026#39;Name\u0026#39;, \u0026#39;Nationality\u0026#39;, \u0026#39;Time\u0026#39;, \u0026#39;Wind\u0026#39;) Let\u0026rsquo;s then create our pandas dataframe with these columns, remembering to skip the first line of the file since it holds the column names we do not want to use:\ndf = pd.read_csv(file, names=columns, skiprows=1) df Gender Event Location Year Medal Name Nationality Time Wind 0 M 10000M Men Rio 2016 G Mohamed FARAH USA 25:05.17 NaN 1 M 10000M Men Rio 2016 S Paul Kipngetich TANUI KEN 27:05.64 NaN 2 M 10000M Men Rio 2016 B Tamirat TOLA ETH 27:06.26 NaN 3 M 10000M Men Beijing 2008 G Kenenisa BEKELE ETH 27:01.17 NaN 4 M 10000M Men Beijing 2008 S Sileshi SIHINE ETH 27:02.77 NaN ... ... ... ... ... ... ... ... ... ... 2389 W Triple Jump Women Athens 2004 S Hrysopiyi DEVETZI GRE 15.25 NaN 2390 W Triple Jump Women Athens 2004 B Tatyana LEBEDEVA RUS 15.14 NaN 2391 W Triple Jump Women Atlanta 1996 G Inessa KRAVETS UKR 15.33 NaN 2392 W Triple Jump Women Atlanta 1996 S Inna LASOVSKAYA RUS 14.98 NaN 2393 W Triple Jump Women Atlanta 1996 B Sarka KASPARKOVA CZE 14.98 NaN 2394 rows × 9 columns\nThe structure of the dataframe seems quite self-explanatory. But we can already notice some details. The format of the athletes\u0026rsquo; times is not consistent. In the first rows we see the format minutes:seconds.hundredths and, in the last rows, we see only seconds.hundredths. Of course, from a sporting point of view it makes no sense to show minutes in short events, but this can make the analysis of the times harder if we decide to do an analysis involving more than one type of event. This is not necessarily a problem, just an observation.\nAnother observation is that there is redundancy in the gender information, since there is a column for it and the event names also include it.\nTo deepen our initial understanding of the data, let\u0026rsquo;s use the info and describe methods. Both are used to get information about the dataframe:\ndf.info() \u0026lt;class 'pandas.core.frame.DataFrame'\u0026gt; RangeIndex: 2394 entries, 0 to 2393 Data columns (total 9 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Gender 2394 non-null object 1 Event 2394 non-null object 2 Location 2394 non-null object 3 Year 2394 non-null int64 4 Medal 2394 non-null object 5 Name 2164 non-null object 6 Nationality 2394 non-null object 7 Time 2394 non-null object 8 Wind 12 non-null float64 dtypes: float64(1), int64(1), object(7) memory usage: 168.5+ KB With info we see that Pandas recognized the year column as integer and the wind column as float. All the others are of type object which, according to the package documentation, is the type for columns with strings or columns with mixed types. The time column being of type object will have consequences later in the article.\nThe Wind column has only 12 non-null values, indicating that a good part of the records lack this information. It may be that there really was no wind in those cases, that is, it would be equivalent to zero, or that the information is missing from the database for some reason.\nThere are also null records in the Name column, let\u0026rsquo;s see how this reflects in our analysis throughout the article.\ndf.describe() Year Wind count 2394.000000 12.00000 mean 1970.379282 -0.02500 std 34.711777 0.56909 min 1896.000000 -0.90000 25% 1948.000000 -0.15000 50% 1976.000000 0.10000 75% 2000.000000 0.22500 max 2016.000000 0.60000 The describe method, when used without any parameter, presents a brief statistical summary of the numeric columns. In this case, the year and wind columns. Note that the data apparently do go from 1896 to 2016, since these are the minimum and maximum values, respectively. But that does not guarantee that every Olympic year in this interval is present. To check:\ndf[\u0026#39;Year\u0026#39;].unique() array([2016, 2008, 2000, 1992, 1984, 1976, 1968, 1960, 1952, 1936, 1928, 1920, 2012, 2004, 1996, 1980, 1972, 1964, 1956, 1948, 1932, 1924, 1912, 1908, 1900, 1904, 1896, 1988]) The unique method shows the unique values in a given column. Note that the years are out of order, which makes the analysis a bit harder. Let\u0026rsquo;s sort them in ascending order with the sort_values method:\ndf[\u0026#39;Year\u0026#39;].sort_values().unique() array([1896, 1900, 1904, 1908, 1912, 1920, 1924, 1928, 1932, 1936, 1948, 1952, 1956, 1960, 1964, 1968, 1972, 1976, 1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008, 2012, 2016]) Note that the years 1916, 1940 and 1944 are missing from this list. But that makes sense, those are world war years\u0026hellip; with no Olympics.\nThe describe method can take parameters, among them include, which allows us to force the presence of other types. Let\u0026rsquo;s see a description of the object types:\ndf.describe(include=\u0026#39;object\u0026#39;) Gender Event Location Medal Name Nationality Time count 2394 2394 2394 2394 2164 2394 2394 unique 2 47 23 3 1681 97 1947 top M Marathon Men London B Merlene OTTEY USA None freq 1632 84 253 799 7 639 43 Note that the information shown has changed. The count row is self-explanatory and shows us something we already had from info. The unique row is similar to the method we used earlier, showing how many unique values there are in each column. The values for gender and medals are easy to understand. Besides that, we find that there are 47 events in the dataset, that 23 locations have hosted the Olympics, that athletes from 97 different nationalities have won medals in track and field (at least up to 2016) and that 2164 - 1681 = 483 athletes won more than one medal.\nThe top row shows the values that repeated the most, while the freq row shows how many times the values in the top row repeated. Thus, we see that the country that won the most medals was the United States.\nBut we need to be careful with the way the top and freq rows are presented, because if more than one entry occurs the same number of times, only one will be shown. Let\u0026rsquo;s look for the athlete who won the most medals in a more complete way. To do that, we show the beginning of the medal column, in descending order, after grouping the data by athlete name:\ndf.groupby(by=\u0026#39;Name\u0026#39;).count()[\u0026#39;Medal\u0026#39;].sort_values(ascending=False).head() Name Merlene OTTEY 7 Paavo NURMI 7 Tirunesh DIBABA 6 Usain BOLT 6 Irena KIRSZENSTEIN 6 Name: Medal, dtype: int64 We see that two athletes won 7 medals in track and field according to the dataset: Paavo Nurmi and Merlene Ottey, both well worth reading about. And Bolt also appears among the top ones, an athlete already covered here on the site. However, these numbers do not exactly match those that appear on the links I placed for each of them. This may be a sign of problems with the dataset. It could be, for example, a typo in the athletes\u0026rsquo; names or simply an incomplete dataset. Let\u0026rsquo;s move on, but already aware of these signs of problems.\nNotice that with just a few small commands we were already able to find relevant information. Get used to doing this preliminary analysis with info and describe. It helps a lot to understand how the data is presented, to get an idea of its completeness and consistency, and to guide later analyses.\nNow is the moment to choose what we want to do. There are several possibilities but, for this article, I decided to focus on the women\u0026rsquo;s 100 m. After all, I have already written an article about Usain Bolt\u0026rsquo;s data so let\u0026rsquo;s give the women some space too, especially with the Olympic record broken now in 2020.\nFocusing on one event # To filter the data we can use the loc method, searching the Event column for 100M Women:\ndf.loc[df[\u0026#39;Event\u0026#39;] == \u0026#39;100M Women\u0026#39;] Gender Event Location Year Medal Name Nationality Time Wind 1687 W 100M Women Rio 2016 G Elaine THOMPSON JAM 10.71 NaN 1688 W 100M Women Rio 2016 S Tori BOWIE USA 10.83 NaN 1689 W 100M Women Rio 2016 B Shelly-Ann FRASER-PRYCE JAM 10.86 NaN 1690 W 100M Women Beijing 2008 G Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1691 W 100M Women Beijing 2008 S Sherone SIMPSON JAM 10.98 NaN 1692 W 100M Women Beijing 2008 S Kerron STEWART JAM 10.98 NaN 1693 W 100M Women Sydney 2000 S Ekaterini THANOU GRE 11.12 NaN 1694 W 100M Women Sydney 2000 S Tanya LAWRENCE JAM 11.18 NaN 1695 W 100M Women Sydney 2000 B Merlene OTTEY JAM 11.19 NaN 1696 W 100M Women Barcelona 1992 G Gail DEVERS USA 10.82 NaN 1697 W 100M Women Barcelona 1992 S Juliet CUTHBERT JAM 10.83 NaN 1698 W 100M Women Barcelona 1992 B Irina PRIVALOVA EUN 10.84 NaN 1699 W 100M Women Los Angeles 1984 G Evelyn ASHFORD USA 10.97 NaN 1700 W 100M Women Los Angeles 1984 S Alice BROWN USA 11.13 NaN 1701 W 100M Women Los Angeles 1984 B Merlene OTTEY JAM 11.16 NaN 1702 W 100M Women Montreal 1976 G Annegret RICHTER-IRRGANG FRG 11.08 NaN 1703 W 100M Women Montreal 1976 S Renate STECHER GDR 11.13 NaN 1704 W 100M Women Montreal 1976 B Inge HELTEN FRG 11.17 NaN 1705 W 100M Women Mexico 1968 G Wyomia TYUS USA 11.0 NaN 1706 W 100M Women Mexico 1968 S Barbara FERRELL USA 11.1 NaN 1707 W 100M Women Mexico 1968 B Irena KIRSZENSTEIN POL 11.1 NaN 1708 W 100M Women Rome 1960 G Wilma RUDOLPH USA 11.0 NaN 1709 W 100M Women Rome 1960 S Dorothy HYMAN GBR 11.3 NaN 1710 W 100M Women Rome 1960 B Giuseppina LEONE ITA 11.3 NaN 1711 W 100M Women Helsinki 1952 G Marjorie JACKSON AUS 11.5 NaN 1712 W 100M Women Helsinki 1952 S Daphne HASENJAGER RSA 11.8 NaN 1713 W 100M Women Helsinki 1952 B Shirley STRICKLAND AUS 11.9 NaN 1714 W 100M Women Berlin 1936 G Helen STEPHENS USA 11.5 NaN 1715 W 100M Women Berlin 1936 S Stanislawa WALASIEWICZ POL 11.7 NaN 1716 W 100M Women Berlin 1936 B Käthe KRAUSS GER 11.9 NaN 1717 W 100M Women Amsterdam 1928 G Elizabeth ROBINSON USA 12.2 NaN 1718 W 100M Women Amsterdam 1928 S Fanny ROSENFELD CAN None NaN 1719 W 100M Women Amsterdam 1928 B Ethel SMITH CAN None NaN 1720 W 100M Women London 2012 G Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1721 W 100M Women London 2012 S Carmelita JETER USA 10.78 NaN 1722 W 100M Women London 2012 B Veronica CAMPBELL-BROWN JAM 10.81 NaN 1723 W 100M Women Athens 2004 G Yuliya NESTSIARENKA BLR 10.93 NaN 1724 W 100M Women Athens 2004 S Lauryn WILLIAMS USA 10.96 NaN 1725 W 100M Women Athens 2004 B Veronica CAMPBELL-BROWN JAM 10.97 NaN 1726 W 100M Women Atlanta 1996 G Gail DEVERS USA 10.94 NaN 1727 W 100M Women Atlanta 1996 S Merlene OTTEY JAM 10.94 NaN 1728 W 100M Women Atlanta 1996 B Gwen TORRENCE USA 10.96 NaN 1729 W 100M Women Moscow 1980 G Lyudmila KONDRATYEVA URS 11.06 NaN 1730 W 100M Women Moscow 1980 S Marlies OELSNER-GÖHR GDR 11.07 NaN 1731 W 100M Women Moscow 1980 B Ingrid AUERSWALD-LANGE GDR 11.14 NaN 1732 W 100M Women Munich 1972 G Renate STECHER GDR 11.07 NaN 1733 W 100M Women Munich 1972 S Raelene Ann BOYLE AUS 11.23 NaN 1734 W 100M Women Munich 1972 B Silvia CHIVAS BARO CUB 11.24 NaN 1735 W 100M Women Tokyo 1964 G Wyomia TYUS USA 11.4 NaN 1736 W 100M Women Tokyo 1964 S Edith MCGUIRE USA 11.6 NaN 1737 W 100M Women Tokyo 1964 B Ewa KLOBUKOWSKA POL 11.6 NaN 1738 W 100M Women Melbourne / Stockholm 1956 G Betty CUTHBERT AUS 11.5 NaN 1739 W 100M Women Melbourne / Stockholm 1956 S Christa STUBNICK EUA 11.7 NaN 1740 W 100M Women Melbourne / Stockholm 1956 B Marlene MATHEWS-WILLARD AUS 11.7 NaN 1741 W 100M Women London 1948 G Fanny BLANKERS-KOEN NED 11.9 NaN 1742 W 100M Women London 1948 S Dorothy HALL GBR 12.2 NaN 1743 W 100M Women London 1948 B Shirley STRICKLAND AUS 12.2 NaN 1744 W 100M Women Los Angeles 1932 G Stanislawa WALASIEWICZ POL 11.9 NaN 1745 W 100M Women Los Angeles 1932 S Hilda STRIKE CAN 11.9 NaN 1746 W 100M Women Los Angeles 1932 B Wilhelmina VON BREMEN USA 12.0 NaN Now we need to choose again what to do, which path of analysis to take. We could, for example, evaluate the relative distance between the gold, silver and bronze medalists to check whether the times are getting closer over the years. Or evaluate the gain/loss of time between each edition of the event. There are many possibilities. Here, I will choose a relatively simple analysis, so that the article is an introduction to pandas. We will look at the evolution of the gold medalists\u0026rsquo; times from the first Olympics that had the women\u0026rsquo;s 100 m competition up to the current edition.\nTherefore, we need to add one more condition to our filter, searching for G (for gold) in the Medal column. The \u0026amp; symbol indicates that both conditions must be satisfied:\ndf.loc[(df[\u0026#39;Event\u0026#39;] == \u0026#39;100M Women\u0026#39;) \u0026amp; (df[\u0026#39;Medal\u0026#39;] == \u0026#39;G\u0026#39;)] Gender Event Location Year Medal Name Nationality Time Wind 1687 W 100M Women Rio 2016 G Elaine THOMPSON JAM 10.71 NaN 1690 W 100M Women Beijing 2008 G Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1696 W 100M Women Barcelona 1992 G Gail DEVERS USA 10.82 NaN 1699 W 100M Women Los Angeles 1984 G Evelyn ASHFORD USA 10.97 NaN 1702 W 100M Women Montreal 1976 G Annegret RICHTER-IRRGANG FRG 11.08 NaN 1705 W 100M Women Mexico 1968 G Wyomia TYUS USA 11.0 NaN 1708 W 100M Women Rome 1960 G Wilma RUDOLPH USA 11.0 NaN 1711 W 100M Women Helsinki 1952 G Marjorie JACKSON AUS 11.5 NaN 1714 W 100M Women Berlin 1936 G Helen STEPHENS USA 11.5 NaN 1717 W 100M Women Amsterdam 1928 G Elizabeth ROBINSON USA 12.2 NaN 1720 W 100M Women London 2012 G Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1723 W 100M Women Athens 2004 G Yuliya NESTSIARENKA BLR 10.93 NaN 1726 W 100M Women Atlanta 1996 G Gail DEVERS USA 10.94 NaN 1729 W 100M Women Moscow 1980 G Lyudmila KONDRATYEVA URS 11.06 NaN 1732 W 100M Women Munich 1972 G Renate STECHER GDR 11.07 NaN 1735 W 100M Women Tokyo 1964 G Wyomia TYUS USA 11.4 NaN 1738 W 100M Women Melbourne / Stockholm 1956 G Betty CUTHBERT AUS 11.5 NaN 1741 W 100M Women London 1948 G Fanny BLANKERS-KOEN NED 11.9 NaN 1744 W 100M Women Los Angeles 1932 G Stanislawa WALASIEWICZ POL 11.9 NaN OK, the filter seems to make sense, so let\u0026rsquo;s assign this dataframe to a variable so that we can do further analyses:\ndf_women_100m = df.loc[(df[\u0026#39;Event\u0026#39;] == \u0026#39;100M Women\u0026#39;) \u0026amp; (df[\u0026#39;Medal\u0026#39;] == \u0026#39;G\u0026#39;)] df_women_100m Gender Event Location Year Medal Name Nationality Time Wind 1687 W 100M Women Rio 2016 G Elaine THOMPSON JAM 10.71 NaN 1690 W 100M Women Beijing 2008 G Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1696 W 100M Women Barcelona 1992 G Gail DEVERS USA 10.82 NaN 1699 W 100M Women Los Angeles 1984 G Evelyn ASHFORD USA 10.97 NaN 1702 W 100M Women Montreal 1976 G Annegret RICHTER-IRRGANG FRG 11.08 NaN 1705 W 100M Women Mexico 1968 G Wyomia TYUS USA 11.0 NaN 1708 W 100M Women Rome 1960 G Wilma RUDOLPH USA 11.0 NaN 1711 W 100M Women Helsinki 1952 G Marjorie JACKSON AUS 11.5 NaN 1714 W 100M Women Berlin 1936 G Helen STEPHENS USA 11.5 NaN 1717 W 100M Women Amsterdam 1928 G Elizabeth ROBINSON USA 12.2 NaN 1720 W 100M Women London 2012 G Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1723 W 100M Women Athens 2004 G Yuliya NESTSIARENKA BLR 10.93 NaN 1726 W 100M Women Atlanta 1996 G Gail DEVERS USA 10.94 NaN 1729 W 100M Women Moscow 1980 G Lyudmila KONDRATYEVA URS 11.06 NaN 1732 W 100M Women Munich 1972 G Renate STECHER GDR 11.07 NaN 1735 W 100M Women Tokyo 1964 G Wyomia TYUS USA 11.4 NaN 1738 W 100M Women Melbourne / Stockholm 1956 G Betty CUTHBERT AUS 11.5 NaN 1741 W 100M Women London 1948 G Fanny BLANKERS-KOEN NED 11.9 NaN 1744 W 100M Women Los Angeles 1932 G Stanislawa WALASIEWICZ POL 11.9 NaN Checking the completeness of the dataset # Before doing more analyses on top of the data, let\u0026rsquo;s check whether it is complete and seemingly coherent. We saw earlier that the years 1916, 1940 and 1944 are not present in the dataset because there were no Olympics in those years. But let\u0026rsquo;s see whether the dataset is complete for the women\u0026rsquo;s 100 m. After all, earlier we did not have such a granular analysis, event by event, so some might be incomplete without our being able to know.\nFollowing the same procedure as before, let\u0026rsquo;s sort the year column:\ndf_women_100m.sort_values(by=\u0026#39;Year\u0026#39;) Gender Event Location Year Medal Name Nationality Time Wind 1717 W 100M Women Amsterdam 1928 G Elizabeth ROBINSON USA 12.2 NaN 1744 W 100M Women Los Angeles 1932 G Stanislawa WALASIEWICZ POL 11.9 NaN 1714 W 100M Women Berlin 1936 G Helen STEPHENS USA 11.5 NaN 1741 W 100M Women London 1948 G Fanny BLANKERS-KOEN NED 11.9 NaN 1711 W 100M Women Helsinki 1952 G Marjorie JACKSON AUS 11.5 NaN 1738 W 100M Women Melbourne / Stockholm 1956 G Betty CUTHBERT AUS 11.5 NaN 1708 W 100M Women Rome 1960 G Wilma RUDOLPH USA 11.0 NaN 1735 W 100M Women Tokyo 1964 G Wyomia TYUS USA 11.4 NaN 1705 W 100M Women Mexico 1968 G Wyomia TYUS USA 11.0 NaN 1732 W 100M Women Munich 1972 G Renate STECHER GDR 11.07 NaN 1702 W 100M Women Montreal 1976 G Annegret RICHTER-IRRGANG FRG 11.08 NaN 1729 W 100M Women Moscow 1980 G Lyudmila KONDRATYEVA URS 11.06 NaN 1699 W 100M Women Los Angeles 1984 G Evelyn ASHFORD USA 10.97 NaN 1696 W 100M Women Barcelona 1992 G Gail DEVERS USA 10.82 NaN 1726 W 100M Women Atlanta 1996 G Gail DEVERS USA 10.94 NaN 1723 W 100M Women Athens 2004 G Yuliya NESTSIARENKA BLR 10.93 NaN 1690 W 100M Women Beijing 2008 G Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1720 W 100M Women London 2012 G Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1687 W 100M Women Rio 2016 G Elaine THOMPSON JAM 10.71 NaN Some interesting things already appear. The first edition in the dataset is 1928 and this is correct. The first women\u0026rsquo;s edition really was in 1928, while the men\u0026rsquo;s has taken place since 1896.\nNow, there are two strange absences: 1988 and 2000. Let\u0026rsquo;s look for those years in the full dataframe, starting with the year 2000:\ndf.loc[(df[\u0026#39;Year\u0026#39;] == 2000) \u0026amp; (df[\u0026#39;Event\u0026#39;] == \u0026#39;100M Women\u0026#39;)] Gender Event Location Year Medal Name Nationality Time Wind 1693 W 100M Women Sydney 2000 S Ekaterini THANOU GRE 11.12 NaN 1694 W 100M Women Sydney 2000 S Tanya LAWRENCE JAM 11.18 NaN 1695 W 100M Women Sydney 2000 B Merlene OTTEY JAM 11.19 NaN Note that there is, indeed, a record of the event in the dataset, but without a gold medalist. And, strange as it may seem, this is also correct. The gold medalist would have been Marion Jones but, due to the use of substances banned in the sport, she ended up losing the medal and the International Olympic Committee decided not to promote Ekaterini Thanou, given that she too had been involved in doping cases. It is a long story that can be read at the links I left here.\nWhat matters here is that the dataset is coherent on this point. It remains to decide what to do with this information. Since we chose to filter only gold medalists, the year 2000 would indeed be left without a time record. Another approach would be to consider Thanou\u0026rsquo;s time, since it would officially be the best time of that year\u0026rsquo;s final, perhaps adding some visual indication that she is not a gold medalist. It all depends on the story you want to tell and the kind of analysis you want to do. Here, I will keep the decision to include only gold medalists and, therefore, the year 2000 will be left without a record.\nLet\u0026rsquo;s evaluate the year 1988 now:\ndf.loc[(df[\u0026#39;Year\u0026#39;] == 1988) \u0026amp; (df[\u0026#39;Event\u0026#39;] == \u0026#39;100M Women\u0026#39;)] Gender Event Location Year Medal Name Nationality Time Wind Here the case is different, there is no record of the event for 1988, which is wrong. It may be that this year is compromised in the dataset, let\u0026rsquo;s check:\ndf.loc[(df[\u0026#39;Year\u0026#39;] == 1988)] Gender Event Location Year Medal Name Nationality Time Wind 423 M 20Km Race Walk Men Seoul 1988 G Jozef PRIBILINEC TCH 1:19:57 NaN 424 M 20Km Race Walk Men Seoul 1988 S Ronald WEIGEL GDR 1:20:00 NaN 425 M 20Km Race Walk Men Seoul 1988 B Maurizio DAMILANO ITA 1:20:14 NaN 911 M 50Km Race Walk Men Seoul 1988 G Vyacheslav IVANENKO URS 3:38:29 NaN 912 M 50Km Race Walk Men Seoul 1988 S Ronald WEIGEL GDR 3:38:56 NaN 913 M 50Km Race Walk Men Seoul 1988 B Hartwig GAUDER GDR 3:39:45 NaN 1058 M Decathlon Men Seoul 1988 G Christian SCHENK GDR 8488.0 NaN 1059 M Decathlon Men Seoul 1988 S Torsten VOSS GDR 8399.0 NaN 1060 M Decathlon Men Seoul 1988 B Dave STEEN CAN 8328.0 NaN 1426 M Marathon Men Seoul 1988 G Gelindo BORDIN ITA 2:10:32 NaN 1427 M Marathon Men Seoul 1988 S Douglas WAKIIHURI KEN 2:10:47 NaN 1428 M Marathon Men Seoul 1988 B Hussein AHMED SALAH DJI 2:10:59 NaN 2162 W Heptathlon Women Seoul 1988 G Jackie JOYNER USA 7291 P. NaN 2163 W Heptathlon Women Seoul 1988 S Sabine JOHN GDR 6897.0 NaN 2164 W Heptathlon Women Seoul 1988 B Anke BEHMER GDR 6858.0 NaN 2319 W Marathon Women Seoul 1988 G Rosa MOTA POR 2:25:40 NaN 2320 W Marathon Women Seoul 1988 S Lisa ONDIEKI AUS 2:25:53 NaN 2321 W Marathon Women Seoul 1988 B Katrin DÖRRE GDR 2:26:21 NaN There really is a problem in the dataset, all the short-distance events are missing. And, unfortunately, there is no warning about it on the dataset\u0026rsquo;s Kaggle page. That is why it is important to do these coherence tests. Note that even if the data came directly from the official Olympics website this kind of check is essential, because errors happen. Here, it may be that the site did not have the data at the time of collection or that there was some error in the algorithm used to scrape the data. Anyway, countless possibilities; what matters is that we noticed the problem and it is simple to fix.\nA quick search on the Olympics website itself shows that the 1988 gold medalist was Florence Griffith Joyner, another athlete whose story is interesting (and short\u0026hellip;). With the data from the site, we can update our dataframe, also taking the opportunity to add the 2020 gold medalist, Elaine Thompson. To do that, we will use the loc method, adding rows after the existing records:\ndf_women_100m.loc[len(df_women_100m.index)] = [\u0026#39;W\u0026#39;, \u0026#39;100M Women\u0026#39;, \u0026#39;Seoul\u0026#39;, 1988, \u0026#39;G\u0026#39;, \u0026#39;Florence Griffith JOYNER\u0026#39;, \u0026#39;USA\u0026#39;, 10.54, 3.0] df_women_100m.loc[len(df_women_100m.index)] = [\u0026#39;W\u0026#39;, \u0026#39;100M Women\u0026#39;, \u0026#39;Tokyo\u0026#39;, 2020, \u0026#39;G\u0026#39;, \u0026#39;Elaine THOMPSON\u0026#39;, \u0026#39;JAM\u0026#39;, 10.61, -0.6] df_women_100m Gender Event Location Year Medal Name Nationality Time Wind 1687 W 100M Women Rio 2016 G Elaine THOMPSON JAM 10.71 NaN 1690 W 100M Women Beijing 2008 G Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1696 W 100M Women Barcelona 1992 G Gail DEVERS USA 10.82 NaN 1699 W 100M Women Los Angeles 1984 G Evelyn ASHFORD USA 10.97 NaN 1702 W 100M Women Montreal 1976 G Annegret RICHTER-IRRGANG FRG 11.08 NaN 1705 W 100M Women Mexico 1968 G Wyomia TYUS USA 11.0 NaN 1708 W 100M Women Rome 1960 G Wilma RUDOLPH USA 11.0 NaN 1711 W 100M Women Helsinki 1952 G Marjorie JACKSON AUS 11.5 NaN 1714 W 100M Women Berlin 1936 G Helen STEPHENS USA 11.5 NaN 1717 W 100M Women Amsterdam 1928 G Elizabeth ROBINSON USA 12.2 NaN 1720 W 100M Women London 2012 G Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1723 W 100M Women Athens 2004 G Yuliya NESTSIARENKA BLR 10.93 NaN 1726 W 100M Women Atlanta 1996 G Gail DEVERS USA 10.94 NaN 1729 W 100M Women Moscow 1980 G Lyudmila KONDRATYEVA URS 11.06 NaN 1732 W 100M Women Munich 1972 G Renate STECHER GDR 11.07 NaN 1735 W 100M Women Tokyo 1964 G Wyomia TYUS USA 11.4 NaN 1738 W 100M Women Melbourne / Stockholm 1956 G Betty CUTHBERT AUS 11.5 NaN 1741 W 100M Women London 1948 G Fanny BLANKERS-KOEN NED 11.9 NaN 1744 W 100M Women Los Angeles 1932 G Stanislawa WALASIEWICZ POL 11.9 NaN 19 W 100M Women Seoul 1988 G Florence Griffith JOYNER USA 10.54 3.0 20 W 100M Women Tokyo 2020 G Elaine THOMPSON JAM 10.61 -0.6 Here we notice something interesting. Joyner\u0026rsquo;s time in 1988 was better than Thompson\u0026rsquo;s now in 2020. However, since Joyner had a large tailwind of 3.0 m/s, it was not considered an Olympic record. The previous Olympic record was Joyner\u0026rsquo;s own, with a time of 10.62 s set in the quarterfinals of the 1988 Olympics with a tailwind of 1.0 m/s and, therefore, within the limit accepted for record purposes.\nBy the way, a brief tangent about wind. The world record to this day is again Florence Joyner\u0026rsquo;s, with a time of 10.49 s set at the trials for the 1988 Olympics. The wind officially recorded during the race was 0 m/s. However, Joyner\u0026rsquo;s time was so striking that people took a closer look at the records of the whole event that day. In several events, winds of around 5.0 m/s were recorded, so it is believed that there was a failure in the anemometer (an instrument that measures the speed of a fluid) used in the race. In any case, the record was kept and, even if it were removed, the second best time is still Joyner\u0026rsquo;s, with 10.61 s without wind assistance. That time has now been equaled by Thompson.\nAgain, here we have something to think about. The goal is to show the evolution of the gold medalists\u0026rsquo; times. From this standpoint, Joyner\u0026rsquo;s time is better, but there was wind assistance. It is up to whoever is telling the story to decide how to handle this situation. Whether to make it evident in some way, or to replace it with the best unassisted time. Anyway, choices to be made, and we will deal with them ahead.\nTidying up the dataframe # Now that we are confident in the consistency of our data and have learned more about it, let\u0026rsquo;s tidy up. We can remove redundant columns:\ndf_women_100m = df_women_100m.drop(columns=[\u0026#39;Gender\u0026#39;, \u0026#39;Event\u0026#39;, \u0026#39;Medal\u0026#39;]) df_women_100m Location Year Name Nationality Time Wind 1687 Rio 2016 Elaine THOMPSON JAM 10.71 NaN 1690 Beijing 2008 Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 1696 Barcelona 1992 Gail DEVERS USA 10.82 NaN 1699 Los Angeles 1984 Evelyn ASHFORD USA 10.97 NaN 1702 Montreal 1976 Annegret RICHTER-IRRGANG FRG 11.08 NaN 1705 Mexico 1968 Wyomia TYUS USA 11.0 NaN 1708 Rome 1960 Wilma RUDOLPH USA 11.0 NaN 1711 Helsinki 1952 Marjorie JACKSON AUS 11.5 NaN 1714 Berlin 1936 Helen STEPHENS USA 11.5 NaN 1717 Amsterdam 1928 Elizabeth ROBINSON USA 12.2 NaN 1720 London 2012 Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 1723 Athens 2004 Yuliya NESTSIARENKA BLR 10.93 NaN 1726 Atlanta 1996 Gail DEVERS USA 10.94 NaN 1729 Moscow 1980 Lyudmila KONDRATYEVA URS 11.06 NaN 1732 Munich 1972 Renate STECHER GDR 11.07 NaN 1735 Tokyo 1964 Wyomia TYUS USA 11.4 NaN 1738 Melbourne / Stockholm 1956 Betty CUTHBERT AUS 11.5 NaN 1741 London 1948 Fanny BLANKERS-KOEN NED 11.9 NaN 1744 Los Angeles 1932 Stanislawa WALASIEWICZ POL 11.9 NaN 19 Seoul 1988 Florence Griffith JOYNER USA 10.54 3.0 20 Tokyo 2020 Elaine THOMPSON JAM 10.61 -0.6 And also fix the index of the records, which still refers to row numbers of the original full dataframe, and take the chance to leave the dataframe sorted by year to make it easier to understand:\ndf_women_100m = df_women_100m.sort_values(by=\u0026#39;Year\u0026#39;).reset_index(drop=True) df_women_100m Location Year Name Nationality Time Wind 0 Amsterdam 1928 Elizabeth ROBINSON USA 12.2 NaN 1 Los Angeles 1932 Stanislawa WALASIEWICZ POL 11.9 NaN 2 Berlin 1936 Helen STEPHENS USA 11.5 NaN 3 London 1948 Fanny BLANKERS-KOEN NED 11.9 NaN 4 Helsinki 1952 Marjorie JACKSON AUS 11.5 NaN 5 Melbourne / Stockholm 1956 Betty CUTHBERT AUS 11.5 NaN 6 Rome 1960 Wilma RUDOLPH USA 11.0 NaN 7 Tokyo 1964 Wyomia TYUS USA 11.4 NaN 8 Mexico 1968 Wyomia TYUS USA 11.0 NaN 9 Munich 1972 Renate STECHER GDR 11.07 NaN 10 Montreal 1976 Annegret RICHTER-IRRGANG FRG 11.08 NaN 11 Moscow 1980 Lyudmila KONDRATYEVA URS 11.06 NaN 12 Los Angeles 1984 Evelyn ASHFORD USA 10.97 NaN 13 Seoul 1988 Florence Griffith JOYNER USA 10.54 3.0 14 Barcelona 1992 Gail DEVERS USA 10.82 NaN 15 Atlanta 1996 Gail DEVERS USA 10.94 NaN 16 Athens 2004 Yuliya NESTSIARENKA BLR 10.93 NaN 17 Beijing 2008 Shelly-Ann FRASER-PRYCE JAM 10.78 NaN 18 London 2012 Shelly-Ann FRASER-PRYCE JAM 10.75 NaN 19 Rio 2016 Elaine THOMPSON JAM 10.71 NaN 20 Tokyo 2020 Elaine THOMPSON JAM 10.61 -0.6 From our earlier analysis, we have already seen that the Time column is seen by Pandas as being of type object, which is wrong. We need to turn it into a numeric type to continue our analyses. But, before that, pay a bit more attention to the time data. Do you notice any difference in the way it is presented over time?\nThe records up to 1968 have only one decimal place. A detailed history of the use of automatic timing devices in track and field can be read here but, in short, times with precision to the hundredths only really began to be used in the Olympics in 1972. However, since the 1928 Olympics automatic timers had already been used, together with manual records. At the Olympics of the 1960s the official results were obtained with automatic timers, but officially published rounded to the first decimal place. Only in 1972 did they begin to officially publish values to the hundredths. One more interesting piece of information that can be obtained by looking at how the data was presented.\nPandas has two very useful methods for dealing with time records, to_datetime and to_timedelta which would certainly be necessary to handle the times of longer events, where the clock format Hour:minute:second.millisecond is used. However, in our case, the values are similar enough to the float format that we can risk a direct conversion to that type, since it will certainly be less work:\ndf_women_100m[\u0026#39;Time\u0026#39;].astype(float) 0 12.20 1 11.90 2 11.50 3 11.90 4 11.50 5 11.50 6 11.00 7 11.40 8 11.00 9 11.07 10 11.08 11 11.06 12 10.97 13 10.54 14 10.82 15 10.94 16 10.93 17 10.78 18 10.75 19 10.71 20 10.61 Name: Time, dtype: float64 Great, Pandas managed to convert all the records to float without problems. But notice that we lost the information that the data before 1972 has only one decimal place, because the conversion gives all of them two. That is why a preliminary analysis is important before doing major manipulations, since the preliminary analysis often provides many insights that lead to interesting research.\nLet\u0026rsquo;s save the column data as float:\ndf_women_100m[\u0026#39;Time\u0026#39;] = df_women_100m[\u0026#39;Time\u0026#39;].astype(float) And, to make it easier to understand, let\u0026rsquo;s make the unit of this time explicit, seconds. This will also help us later in the plots.\ndf_women_100m = df_women_100m.rename(columns={\u0026#39;Time\u0026#39;: \u0026#39;Time / s\u0026#39;}) Let\u0026rsquo;s then evaluate info and describe of our dataframe:\ndf_women_100m.info() \u0026lt;class 'pandas.core.frame.DataFrame'\u0026gt; RangeIndex: 21 entries, 0 to 20 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Location 21 non-null object 1 Year 21 non-null int64 2 Name 21 non-null object 3 Nationality 21 non-null object 4 Time / s 21 non-null float64 5 Wind 2 non-null float64 dtypes: float64(2), int64(1), object(3) memory usage: 1.1+ KB df_women_100m.describe() Year Time / s Wind count 21.000000 21.000000 2.000000 mean 1975.809524 11.150476 1.200000 std 27.927798 0.453315 2.545584 min 1928.000000 10.540000 -0.600000 25% 1956.000000 10.820000 0.300000 50% 1976.000000 11.000000 1.200000 75% 1996.000000 11.500000 2.100000 max 2020.000000 12.200000 3.000000 df_women_100m.describe(include=\u0026#39;object\u0026#39;) Location Name Nationality count 21 21 21 unique 18 17 9 top Los Angeles Gail DEVERS USA freq 2 2 9 See that now we have Year and Time as numeric columns, which is what we need for presenting our results. Besides that, no athlete won gold more than twice in this event and the country with the most gold medals is the United States.\nPresenting the results. Images! # Finally, let\u0026rsquo;s get to the plots. Pandas has good integration with Matplotlib, about which I have already written some articles here on the site. Just call the plot method and pass the columns for the x and y axes:\ndf_women_100m.plot(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;) The best way to present this would be a scatter plot, since the lines connecting the periods do not make much sense:\ndf_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;) OK, it is not one of the prettiest plots and a lot could be improved. I have already written an article about customizing plots with Matplotlib and we could go that way. However, it may be more practical to try using ready-made styles:\nimport matplotlib.pyplot as plt plt.style.use(\u0026#39;ggplot\u0026#39;) df_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;) Better, it really looks more presentable and perhaps, for a static media format, such as print, it would be adequate with a few more adjustments. But for a presentation in digital media, more interactivity would be interesting. For that, let\u0026rsquo;s replace the pandas plotting backend with Plotly, an excellent package for building interactive plots:\npd.options.plotting.backend = \u0026#34;plotly\u0026#34; df_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;) We already have a significant visual improvement. But try hovering the mouse (or your finger, if you are on a touch device) over each point. Information about the values on the horizontal and vertical axes appears. Besides that, there is a toolbar at the top of the plot, which lets you zoom into regions and even save the figure. Excellent for digital media, right? And we can still improve. Let\u0026rsquo;s enrich the information that appears on each point, showing the athlete\u0026rsquo;s name and nationality:\ndf_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;, hover_name=\u0026#39;Name\u0026#39;, hover_data=[\u0026#39;Nationality\u0026#39;]) It is still possible to pass more information. We can play with the size of each point and its color. We can tie these two aspects to each athlete\u0026rsquo;s time:\ndf_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;, hover_name=\u0026#39;Name\u0026#39;, hover_data=[\u0026#39;Nationality\u0026#39;], size=\u0026#39;Time / s\u0026#39;, color=\u0026#39;Time / s\u0026#39;) Notice what plotly did. Each circle has an area proportional to the athlete\u0026rsquo;s time. This can be a bit hard to notice since the range of times is small, but look at the 1928 time and the 2020 time and perhaps you will see it. In the video I explain how we could make this difference more evident. But what draws the most attention is that plotly automatically created a color scale that varies gradually with each athlete\u0026rsquo;s time. The level of visual information this adds is high, since the reader identifies that it is something gradual even from a distance and easily finds the maxima and minima.\nEven so, there is room for improvement. Usually these scales are built so that the highest value gets the most striking or \u0026ldquo;hot\u0026rdquo; color. However, in our context, the lower time is the more important one, so it would be nice to be able to change this color scheme. For that, we will import plotly directly. That way, we have access to the themes and color scales.\nAmong the color scales, I really like the one called Viridis. Note that in the code below it was used under the name Viridis_r, the _r is for using the reversed color scale, so that the lowest time gets the most attractive color.\nimport plotly df_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;, hover_name=\u0026#39;Name\u0026#39;, hover_data=[\u0026#39;Nationality\u0026#39;], size=\u0026#39;Time / s\u0026#39;, color=\u0026#39;Time / s\u0026#39;, color_continuous_scale=plotly.colors.sequential.Viridis_r, ) Finally, let\u0026rsquo;s give the plot a title. Always make what you want to show as explicit as possible with a good title and with properly labeled axes.\ndf_women_100m.plot.scatter(x=\u0026#39;Year\u0026#39;, y=\u0026#39;Time / s\u0026#39;, hover_name=\u0026#39;Name\u0026#39;, hover_data=[\u0026#39;Nationality\u0026#39;], size=\u0026#39;Time / s\u0026#39;, color=\u0026#39;Time / s\u0026#39;, color_continuous_scale=plotly.colors.sequential.Viridis_r, title=\u0026#34;Summer Olympics - Women\u0026#39;s 100 m - Gold medalists time progression\u0026#34;) Conclusion # Certainly there is still much to explore in this dataset and other improvements could also be made to our plot. But I believe that a lot has been done in this article and I want to keep it as a simple introduction to Pandas. I will reinforce some of the points discussed:\nobtaining data; checking the quality, completeness and consistency of the data; always do the check, even when the data comes from original sources or was obtained by you, because errors can happen; seek to understand the history and context behind the data; info and describe are very useful; make transformations only when they prove necessary; think about which story can be told; be open to new possibilities for stories; the absence of a data point or an outlier also tells stories; a plot does not live by x and y alone, explore sizes, shapes and colors. I\u0026rsquo;ll leave some ideas here:\nSurely there is some way to indicate that the 1988 time is not the Olympic record. Perhaps a black circle (any color really, as long as it is outside the scale already used), or colorless with a wider border? Or a symbol other than a circle? And, of course, a footnote explaining it; Likewise, Thanou\u0026rsquo;s time could appear as the 2000 time, but with some distinction from the others; The dataset is large, explore it: Has the gap between medalists shrunk over time? Are the contests closer today than in the past? Which events are missing from this dataset? How does the distribution of medals change over time? Has any country that was part of the Soviet Union managed to keep the relevance it had in the USSR era? What is the progression of athletes from underdeveloped or developing countries in these sports? Would it be possible to build a predictive model of the times for the next Olympics? Each of these ideas can be applied to each event and some to the whole set. Have fun and share your explorations in the comments.\nSee you next time.\n","date":"2021-08-08","externalUrl":null,"permalink":"/2021/08/olympics-data-pandas-plotly/","section":"Blog","summary":"The Olympics are coming to an end… and, besides nostalgia and stories of perseverance, every Olympics leaves behind an enormous amount of data! In this article, we will see how to use Pandas to analyze part of that data, looking at how athletes’ times have evolved in track and field events.","title":"Analyzing Olympic Data with Pandas and Plotly","type":"posts"},{"content":"","date":"2021-08-08","externalUrl":null,"permalink":"/tags/pint/","section":"tags","summary":"","title":"Pint","type":"tags"},{"content":"","date":"2021-08-08","externalUrl":null,"permalink":"/tags/plotly/","section":"tags","summary":"","title":"Plotly","type":"tags"},{"content":" Is it possible to study for the college entrance exam and still learn Python? Of course! And that is one of the goals of the Brazilian Exam Problems category on this site. In it, we use computational tools to analyze and expand problems from the ENEM (Exame Nacional do Ensino Médio), the national high school exam that Brazilian students take to enter universities.\nIn today\u0026rsquo;s article, we analyze a problem from the 2019 exam. And, along the way, we learn to make 3D and 4D(?!) plots with Matplotlib.\nThe Jupyter Notebook file with all the code in this article is available at this link. The code in the notebook is in Portuguese, while the code shown here is in English.\nThe problem statement # Below is the statement of the problem, taken from the regular yellow exam given in 2019 (the original is in Portuguese):\nClearly, this is a Physics problem. Let\u0026rsquo;s look at the topics involved:\nfree fall of a brick impulse The physics of the problem # It is nothing new that some ENEM statements leave something to be desired when it comes to the clarity of the information. For example, there is no information about the type of collision (elastic or inelastic?), about the final velocity after the collision (we have to take 0 m/s, but it is a bit unrealistic to assume it stops abruptly), about what counts as the mean impulsive force (the net force, or the force acting opposite to the fall at the moment of impact?), and the \u0026ldquo;can last\u0026rdquo; in the passage the impact can last up to 0.5 s is imprecise. But let\u0026rsquo;s model the problem with what we have.\nConsidering the conservation of mechanical energy, where the final kinetic energy equals the initial potential energy, we have:\n$$ \\frac{m v^2}{2} = mgh \\therefore v = \\sqrt{2gh} $$From this we get the velocity of the brick immediately before the impact.\nFrom the Impulse-Momentum Theorem:\n$$ I = \\Delta Q \\\\ F_m \\cdot \\Delta t = m \\cdot \\Delta v \\therefore F_m = \\frac{m \\cdot \\Delta v}{\\Delta t} $$where I is the impulse, Q is the linear momentum, and Fm is the mean force of the impact.\nSince the problem asks for the answer in terms of the weight of the brick, with N the number of bricks and P the weight of one brick, what is being asked is:\n$$ N = \\frac{F_m}{P} $$Note that, since the mass of the brick appears in both the mean force and the weight, we are in fact also computing the ratio between the (de)celeration of the brick and the acceleration of gravity. We will come back to this point later.\nSolving with Python # Let\u0026rsquo;s start by importing the necessary libraries. Since we want to work with units, we will use the pint library, already famous on this site. For the math, we could use the math library, which is part of the standard Python installation. However, since we will build plots later to expand the problem by varying values, we will use NumPy.\nimport pint import numpy as np The following lines create a unit registry and a quantity constructor, and already configure the library to be used in future Matplotlib plots.\nureg = pint.UnitRegistry() Q_ = ureg.Quantity ureg.setup_matplotlib(True) Now let\u0026rsquo;s assign the values in the statement to variables:\nGRAVITY_ACCELERATION = Q_(\u0026#39;10 m/s^2\u0026#39;) brick_mass = Q_(\u0026#39;2.5 kg\u0026#39;) brick_fall_height = Q_(\u0026#39;5 m\u0026#39;) brick_helmet_impact_time = Q_(\u0026#39;0.5 s\u0026#39;) The great advantage of programming is being able to encapsulate repetitive calculations in functions. So let\u0026rsquo;s define functions for the physical relations we found earlier:\ndef weight_force(mass): return mass * GRAVITY_ACCELERATION def free_fall_velocity(fall_height): return np.sqrt(2 * GRAVITY_ACCELERATION * fall_height) def linear_momentum_change(mass, velocity_change): return mass * velocity_change def mean_impulsive_force(mass, fall_height, impact_time): return linear_momentum_change(mass, free_fall_velocity(fall_height)) / impact_time With these functions and the variables created earlier, we can compute the mean force of the impact:\nmean_impulsive_force(brick_mass, brick_fall_height, brick_helmet_impact_time) 50.0 kilogram meter/second\u0026lt;sup\u0026gt;2\u0026lt;/sup\u0026gt; The unit kg m/s2 is equivalent to the newton (mass times acceleration, right?). Checking with the internal conversion of pint:\nmean_impulsive_force(brick_mass, brick_fall_height, brick_helmet_impact_time).to(\u0026#39;N\u0026#39;) 50.0 newton Notice that we pass only the data from the statement to the function mean_impulsive_force. Internally, the function calls the other functions. This line of thought is fundamental in programming: create specialized functions and take advantage of the idea of composition. Those with more programming experience may even see classes here, but I will leave that for another article.\nNow let\u0026rsquo;s create the function that answers the problem. It needs to divide the mean impulsive force by the weight force. So we can reuse the functions already created and pass only the data from the statement to it:\ndef impact_g_force(mass, fall_height, impact_time): return mean_impulsive_force(mass, fall_height, impact_time) / weight_force(mass) impact_g_force(brick_mass, brick_fall_height, brick_helmet_impact_time) 2.0 dimensionless Since it is a ratio of forces, the result is dimensionless and the answer is (apparently, as we will see later) choice A of the problem. But wait, we are not done yet. Did you see the name I gave this function?\nThe impact g-force # As written earlier in the article, since the answer is a ratio of forces on the same body, it is also the ratio between the acceleration of the body and the acceleration of gravity. Let\u0026rsquo;s see:\nbrick_deceleration = free_fall_velocity(brick_fall_height) / brick_helmet_impact_time brick_deceleration 20.0 meter/second\u0026lt;sup\u0026gt;2\u0026lt;/sup\u0026gt; brick_deceleration / GRAVITY_ACCELERATION 2.0 dimensionless The calculations are in magnitude.\nThe so-called g-force is a measure of force per unit mass, therefore an acceleration, that causes the sensation of weight. The g-force experienced by an object is due to the vector sum of all non-gravitational and non-electromagnetic forces acting on the object, which is free to move.\nIn the case at hand, we see that the value is 2g, with g the acceleration of gravity, hence the value of 20 m/s2.\nThe term force is unfortunate in this case, since it is a measure of acceleration, but the term is so popular that we will use it here too.\nIf you follow motorsport, you have probably seen figures like the one below, showing the g-force on a driver at a given point of a circuit.\nIn the figure, we see Hamilton driving a Mercedes in 2017 at the Australian circuit. We see that in turn 1 of the circuit he was subjected to 6.5g. The full lap can be seen by clicking on the image, which opens a video showing the g-force throughout the lap.\nThink that is little? How about a race that was canceled because of the high accelerations the drivers were subjected to? That is what happened at the Texas oval in 2001. After hearing complaints of severe nausea and dizziness from all the drivers who took part in the practice sessions, the doctors concluded that the banking of the turns subjected the athletes to a force of approximately 5.5g during 18 of the 22 seconds needed to complete a lap. 3g for long periods is considered the limit to preserve a driver\u0026rsquo;s health in races. The video below is from the ESPN coverage of the event at the time:\nThe next two videos show what happens to a person subjected to this level of acceleration for some time:\nNo wonder drivers like Barrichello say the neck is one of the most strained parts of the body in races, since the head is pushed all the time and has to withstand this level of acceleration.\nThis value is impressive, but it is nothing compared to the values that can be reached in crashes. In fact, given the safety of today\u0026rsquo;s racing cars, the biggest risk in crashes is the sudden deceleration, since the driver can suffer internal injuries or pass out. It is no coincidence that we see cars breaking into pieces in crashes. It is a way of trying to dissipate as much energy as possible without causing an abrupt stop. We also have deformable walls and many other technologies to try to minimize the effect of sudden decelerations in crashes.\nEven so, the g-force that can be reached in short time intervals is very high. In Grosjean\u0026rsquo;s recent spectacular crash, shown in the video below, the impact is estimated to have been on the order of 50g. Notice that the car split in two and there was a fire. Fortunately the driver did not pass out despite the incredible deceleration and suffered only light burns on his hands and feet.\nTo end this motorsport tangent, watch the next video of Kenny Brack\u0026rsquo;s crash in 2003. The impact was on the order of 210g. The driver suffered several injuries, which he himself describes in the video. He was able to return to racing in 2005, after 18 months of recovery.\nPics or it didn\u0026rsquo;t happen! Plots and interactivity # If I let myself, I\u0026rsquo;ll keep talking about racing here forever\u0026hellip; Let\u0026rsquo;s now explore the physics of the subject a bit more. Let\u0026rsquo;s create a series of values for height, mass, and impact time, using the NumPy linspace method.\nheight = np.linspace(0.01, 20, 50) * ureg(\u0026#39;m\u0026#39;) mass = np.linspace(0.01, 10, 50) * ureg(\u0026#39;kg\u0026#39;) impact_time = np.linspace(0.01, 0.5, 50) * ureg(\u0026#39;s\u0026#39;) And import the Matplotlib library for the plots.\nimport matplotlib.pyplot as plt For simple plots, such as the variation of the fall velocity with the fall height, this command is enough:\nplt.plot(height, free_fall_velocity(height)) But the plot can be much more expressive, as I have already shown on this site. Let\u0026rsquo;s define a function with plot customizations.\ndef _plot_params(ax=None): \u0026#34;\u0026#34;\u0026#34;Custom parameters for plots. Parameters: ---------- ax : Matplotlib axes, optional, axes where the plot will be drawn, default None Returns: -------- Matplotlib axes, axis for a figure \u0026#34;\u0026#34;\u0026#34; linewidth = 2 size = 12 # grid and ticks ax.minorticks_on() ax.grid(b=True, which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=linewidth - 0.5) ax.grid(b=True, which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=linewidth - 1) ax.tick_params(which=\u0026#39;both\u0026#39;, labelsize=size+2) ax.tick_params(which=\u0026#39;major\u0026#39;, length=6, axis=\u0026#39;both\u0026#39;) ax.tick_params(which=\u0026#39;minor\u0026#39;, length=3, axis=\u0026#39;both\u0026#39;) # axis names ax.xaxis.label.set_size(size+4) ax.yaxis.label.set_size(size+4) return ax Let\u0026rsquo;s create a figure and an associated axis:\nfig1, ax1 = plt.subplots(figsize=(8, 6), facecolor=(1,1,1)) _plot_params(ax1) ax1.plot(height, free_fall_velocity(height)) ax1.set_xlabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{height.units:~P}\u0026#34;) ax1.set_ylabel(\u0026#34;Velocity\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{free_fall_velocity(height).units:~P}\u0026#34;) plt.show() Now we have an informative plot.\nWe can also generate plots for the mean impulsive force and the g-force with the data from the statement.\nfig2, axarr = plt.subplots(ncols=2, nrows=1, facecolor=(1,1,1), figsize=(12, 6), constrained_layout=True) for ax in axarr: _plot_params(ax) axarr[0].plot(height, mean_impulsive_force(brick_mass, height, brick_helmet_impact_time)) axarr[0].set_xlabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{height.units:~P}\u0026#34;) axarr[0].set_ylabel(\u0026#34;Mean impulsive force\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{mean_impulsive_force(brick_mass, height, brick_helmet_impact_time).to(\u0026#39;N\u0026#39;).units:~P}\u0026#34;) axarr[1].plot(height, impact_g_force(brick_mass, height, brick_helmet_impact_time)) axarr[1].set_xlabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{height.units:~P}\u0026#34;) axarr[1].set_ylabel(\u0026#34;g-force\u0026#34;) fig2.suptitle(\u0026#34;Mean impulsive force and g-force for a brick (2.5 kg) in free fall and impact time of 0.5 s\u0026#34;, fontsize=18) plt.show() Now stop and think: both the impulsive force and the g-force depend on the mass, the height, and the impact time. We can make plots in three dimensions. Actually, up to 4 dimensions.\n3D and 4D(?!) plots # Let\u0026rsquo;s start by checking how the g-force varies as a function of mass and height. We need a set of points in two dimensions over which a third will be built. Hence the NumPy meshgrid method:\nMass, Height = np.meshgrid(mass, height) Since the return values are matrices, the variables are named with capital letters, a custom in the scientific community.\nComputing the resulting g-force, still considering the impact time given in the statement:\nG_force = impact_g_force(Mass, Height, brick_helmet_impact_time) Now let\u0026rsquo;s create a surface with the values. Passing the subplot_kw parameter with the instruction to use a 3D projection in the subplots and using the plot_surface method of the created axis, we have:\nfig3, ax3 = plt.subplots(facecolor=(1,1,1), figsize=(8, 6), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) ax3.plot_surface(Mass.magnitude, Height.magnitude, G_force.magnitude) ax3.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;) ax3.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;) ax3.set_zlabel(\u0026#34;g-force\u0026#34;) plt.show() From the surface, we see that the g-force does not depend on the mass, consistent with what we showed earlier.\nAs for viewing options, Matplotlib projections have two angles that can be changed: the elevation angle and the rotation angle of the axis system. Let\u0026rsquo;s illustrate this below with the view_init method. We kept the elevation angle fixed at 30° and changed the rotation angle in steps of 60° from an array of values (angle).\nfig4, axarr4 = plt.subplots(nrows=3, ncols=2, facecolor=(1,1,1), figsize=(10, 10), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) angle = np.arange(60, 361, 60) for i, ax in enumerate(axarr4.flat): ax.plot_surface(Mass.magnitude, Height.magnitude, G_force.magnitude) ax.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;) ax.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;) ax.set_zlabel(\u0026#34;g-force\u0026#34;) ax.view_init(30, angle[i]) plt.subplots_adjust(hspace=0.3) plt.show() Of course, static figures are not very illustrative in this case. It would be better to have an animation with the axis rotating. Let\u0026rsquo;s create one.\nTo do that, we import the animation module and create a function that will take care of our animation. Basically, the function receives an angle and applies it to the currently active axis using plt.gca(), where gca stands for get current axis.\nfrom matplotlib import animation def animate(angle, save_fig=False): ax = plt.gca() # gca = get current axis ax.view_init(30, angle) Since this whole article is being written in a Jupyter Notebook, for the animation to work we need the following special command:\n%matplotlib notebook Now let\u0026rsquo;s create a figure with an axis system and pass the figure to the FuncAnimation method together with the function created for the animation.\nfig5, ax5 = plt.subplots(facecolor=(1,1,1), figsize=(6, 6), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) ax5.plot_surface(Mass.magnitude, Height.magnitude, G_force.magnitude) ax5.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;) ax5.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;) ax5.set_zlabel(\u0026#34;g-force\u0026#34;) anim = animation.FuncAnimation(fig5, animate, frames=361, # 360° rotation interval=10, # interval between frames in milliseconds repeat=False) # if True, it runs indefinitely, like a gif plt.show() Interesting! Now, we know that the g-force also depends on the impact time. In the figure above, the value is fixed at the value in the statement. Let\u0026rsquo;s also create a plot that shows this dependence. First, creating a set of points in 3 dimensions and computing the g-force:\nMass, Height, Impact_time = np.meshgrid(mass, height, impact_time) G_force = impact_g_force(Mass, Height, Impact_time) Let\u0026rsquo;s visualize the set of points we created:\n%matplotlib inline fig6, ax6 = plt.subplots(facecolor=(1,1,1), figsize=(6, 6), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) ax6.scatter(Mass.magnitude, Height.magnitude, Impact_time.magnitude) ax6.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;) ax6.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;) ax6.set_zlabel(\u0026#34;Impact time\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Impact_time.units:~P}\u0026#34;) plt.show() Ok, there is no useful information at all in the figure above, since all the points have the same color. There is no way to add yet another axis, since we are visually limited to understanding up to 3D, but there is a way to add one more dimension of information by making the color of the points meaningful. That way, we get 3D visually but 4D informationally.\nIn a recent article I already used this approach, see how in this article where we study the atmosphere with Python. I will use some ideas from that article here, especially the Matplotlib modules to create a color scale and the idea of normalization:\nimport matplotlib.colors as mcolors import matplotlib.cm as cm The values of the points will be normalized based on G_force and a value scale will be shown next to the projection:\n%matplotlib inline fig7, ax7 = plt.subplots(facecolor=(1,1,1), figsize=(12, 12), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) colors= G_force.magnitude normalize_colors = mcolors.Normalize(vmin=np.min(colors), vmax=np.max(colors)) colormap = cm.YlOrRd img = ax7.scatter(Mass.magnitude, Height.magnitude, Impact_time.magnitude, c=colors, cmap=colormap, norm=normalize_colors) scalarmappaple = cm.ScalarMappable(norm=normalize_colors, cmap=colormap) scalarmappaple.set_array(colors) colorbar = fig7.colorbar(scalarmappaple) ax7.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;) ax7.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;) ax7.set_zlabel(\u0026#34;Impact time\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Impact_time.units:~P}\u0026#34;) colorbar.ax.set_ylabel(\u0026#39;g-force\u0026#39;, rotation=270, labelpad=20, size=15) plt.show() Ok, now we have a bit more information here. We notice that larger g-force values (more intense red colors) are obtained for larger masses and heights and shorter impact times. This is consistent with our mathematical definition, since the impact time is the denominator in the calculation of the impulsive force, so shorter impact times result in a larger impulsive force and, consequently, a larger g-force.\nBut the color distribution could be better, since a good part of the points ended up light yellow. We can use a logarithmic color scale to give more contrast. We just need to change mcolors.Normalize, which creates a scalar normalization, to mcolors.LogNorm, which creates a logarithmic-scale normalization. The rest of the code stays the same:\n%matplotlib inline fig8, ax8 = plt.subplots(facecolor=(1,1,1), figsize=(12, 12), subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;3d\u0026#39;}) colors= G_force.magnitude normalize_colors = mcolors.LogNorm(vmin=np.min(colors), vmax=np.max(colors)) colormap = cm.YlOrRd img = ax8.scatter(Mass.magnitude, Height.magnitude, Impact_time.magnitude, c=colors, cmap=colormap, norm=normalize_colors) scalarmappaple = cm.ScalarMappable(norm=normalize_colors, cmap=colormap) scalarmappaple.set_array(colors) colorbar = fig8.colorbar(scalarmappaple, fraction=0.04, pad=0.05) ax8.set_xlabel(\u0026#34;Mass\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Mass.units:~P}\u0026#34;, fontsize=14) ax8.set_ylabel(\u0026#34;Height\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Height.units:~P}\u0026#34;, fontsize=14) ax8.set_zlabel(\u0026#34;Impact time\u0026#34; + \u0026#34; / \u0026#34; + f\u0026#34;{Impact_time.units:~P}\u0026#34;, fontsize=14) colorbar.ax.set_ylabel(\u0026#39;g-force\u0026#39;, rotation=270, labelpad=20, size=15) ax8.tick_params(axis=\u0026#39;both\u0026#39;, which=\u0026#39;major\u0026#39;, labelsize=13) colorbar.ax.tick_params(labelsize=13) fig8.suptitle(\u0026#34;ENEM and the killer brick problem\u0026#34;, fontsize=20, y=0.85) plt.show() Look at the face of the cube of points facing you and parallel to the mass axis: it is now noticeable that even falls from small heights of low-mass objects can reach considerable g-force values as long as the impact time is very short. The points have a more intense orange color, indicating g-forces already close to 10 times the acceleration of gravity. Of course, the most intense red colors are still for larger heights and masses and shorter impact times.\nThis is an important tip: when you need more contrast between values, try a logarithmic scale.\nOf course, you can now use the previous idea and animate this cube, making it rotate. Just be careful, since there are many points (can you work out how many? Leave it in the comments), so it can be slow. That is left as an exercise. In the future I may write a post about how Matplotlib is slow, but can be optimized.\nThis figure helps make explicit a problem with the statement that I had already mentioned at the beginning of the article. The passage (\u0026hellip;) the impact can last up to 0.5 s. Notice that the imprecision creates a range of g-force values that varies by 4 orders of magnitude (10-1 to 102), depending on the effective impact time. Let\u0026rsquo;s discuss this further.\nCriticisms of the statement # In an excellent article (in Portuguese), the Reference Center for Physics Teaching (CREF) of UFRGS makes harsh criticisms of this problem\u0026rsquo;s statement. As we already pointed out at the beginning of the article, the statement lacks much information. But let\u0026rsquo;s focus on the passage about time.\nSince 0.5 second is the maximum time the impact can last, there is a large range of values the resulting force can have, as we saw in the previous section. So the answer to the problem is at least 2P. Therefore, any choice except A. Why? Well, since time is the denominator in the calculation of the impulsive force, any value smaller than 0.5 second will result in larger forces. The answer key chose the choice that is certainly the most wrong answer\u0026hellip;\nBesides, with this data the brick would certainly go through the helmet! Let\u0026rsquo;s suppose that during the impact the mean relative velocity is about 5 m/s. The brick would penetrate 5 m/s . 0.5 s = 2.5 m into the helmet. That makes no sense. The collision surely has to last much less time for the penetration to be on the order of centimeters. But, as we saw, with less time the impact force is much larger.\nConclusion # So, did you learn more Python and/or more physics? It is a shame that statements like this appear in the ENEM. And there is not much to be done since it is a centralized exam and, directly or indirectly, forced down the throats of the country\u0026rsquo;s educational institutions. But at least we can use these slips to dig deeper into our studies and exercise critical thinking.\nShare this article with students and with anyone else you like. Follow Ciência Programada to know when more articles are released.\nSee you next time!\n","date":"2021-01-08","externalUrl":null,"permalink":"/2021/01/enem-killer-brick-problem/","section":"Blog","summary":"Is it possible to study for the college entrance exam and still learn Python? Of course! And that is one of the goals of the Brazilian Exam Problems category on this site. In it, we use computational tools to analyze and expand problems from the ENEM, Brazil’s national high school exam. In today’s article, we analyze a problem from the 2019 exam. And, along the way, we learn to make 3D and 4D(?!) plots with Matplotlib. All because of a killer brick.","title":"ENEM: The Killer Brick Problem","type":"posts"},{"content":"","date":"2021-01-08","externalUrl":null,"permalink":"/tags/regex/","section":"tags","summary":"","title":"regex","type":"tags"},{"content":"","date":"2021-01-08","externalUrl":null,"permalink":"/tags/web-scraping/","section":"tags","summary":"","title":"web scraping","type":"tags"},{"content":"Pequeno projeto Python onde mostro TDD (desenvolvimento orientado a testes), como criar uma CLI (interface de linha de comando) e como gerenciar um projeto no GitHub.\nOs artigos são listados abaixo, na ordem em que foram escritos.\n","date":"2020-12-29","externalUrl":null,"permalink":"/series/gerenciando-um-pequeno-projeto-python/","section":"Séries","summary":"","title":"Gerenciando um pequeno projeto Python","type":"series"},{"content":"","date":"2020-12-29","externalUrl":null,"permalink":"/tags/github/","section":"tags","summary":"","title":"GitHub","type":"tags"},{"content":"A small Python project in which I show TDD (test-driven development), how to create a CLI (command-line interface) and how to manage a project on GitHub.\nThe articles are listed below, in the order they were written.\n","date":"2020-12-29","externalUrl":null,"permalink":"/series/managing-a-small-python-project/","section":"Series","summary":"","title":"Managing a Small Python Project","type":"series"},{"content":"","date":"2020-12-29","externalUrl":null,"permalink":"/tags/markdown/","section":"tags","summary":"","title":"Markdown","type":"tags"},{"content":"In this video (in Portuguese) I show the basics of Markdown so you can present your project on GitHub.\nLink to the project repository shown in the video.\n","date":"2020-12-29","externalUrl":null,"permalink":"/2020/12/markdown-basics-github-readme/","section":"Blog","summary":"We will see the basics of Markdown for creating a good README on GitHub, with practical examples and tips on how to make your project more attractive.","title":"Markdown Basics for a Good README on GitHub","type":"posts"},{"content":" Colorless, made of invisible constituents, and odorless (we hope!). We often forget that we are surrounded by the gases that make up our atmosphere. In this article, we will see how to easily get a list of the main constituents of air with Python and web scraping. We will also do some calculations, such as density and mean molar mass, and, of course, present everything in spectacular plots.\nHere is a short list of the tools we will use:\nPint ChemPy SciPy pandas Web scraping with pandas Regular expressions Matplotlib ready-made styles color scale scatter plot advanced plot customization Standard Python map anonymous functions (lambda) list comprehensions Lots of useful stuff, so let\u0026rsquo;s go.\nIn physics, chemistry, and engineering courses, situations often come up where we need calculations that depend on characteristics of air. It might be, for example, a combustion reaction happening in the open air, where we need to consider a certain percentage of oxygen in the air, or a compressor, where we need to know the mass of air it holds under certain conditions. Whatever the situation, we usually need the density of air or its mean molar mass. In fact, these quantities are closely related, as we will see.\nInitial setup # To start, let\u0026rsquo;s import some libraries we will use. The role of each one will be detailed along the article. pint has already appeared in other articles on this site, and you can read more about it here.\nfrom scipy.constants import gas_constant import matplotlib.pyplot as plt import pint import chempy import pandas as pd Let\u0026rsquo;s create a unit registry and a quantity constructor from pint and also configure the package to work with plots created via matplotlib.\nureg = pint.UnitRegistry() ureg.setup_matplotlib(True) Q_ = ureg.Quantity Theoretical background and getting chemical data # Now let\u0026rsquo;s get to know the chempy package a little, whose project site is this one. With chempy we can easily get information about a compound from its formula. Let\u0026rsquo;s start with hydrogen gas as an example:\nhydrogen_gas = chempy.Substance.from_formula(\u0026#39;H2\u0026#39;) The code above, quite self-explanatory, instantiates a Substance from the formula as a string.\nLet\u0026rsquo;s look at the attributes and methods publicly available on the hydrogen_gas object we created:\n[x for x in dir(hydrogen_gas) if not x.startswith(\u0026#39;_\u0026#39;)] ['attrs', 'charge', 'composition', 'composition_keys', 'data', 'from_formula', 'html_name', 'latex_name', 'mass', 'molar_mass', 'name', 'unicode_name'] It is interesting to have the option of exporting the formula in formats such as LaTeX and Unicode:\nhydrogen_gas.latex_name 'H_{2}' hydrogen_gas.unicode_name 'H₂' But, for our purposes here, the important thing is to have the molar mass value. We can access it in two ways, through the mass attribute and the molar_mass method:\nhydrogen_gas.mass 2.016 hydrogen_gas.molar_mass() array(2.016) * g/mol As we can see, mass returns a dimensionless float, while molar_mass returns it with units. We can separate the magnitude from the unit if desired:\nhydrogen_gas.molar_mass().magnitude array(2.016) hydrogen_gas.molar_mass().units array(1.) * g/mol Working with units is always desirable. However, chempy does not use pint to manage units. Instead, it uses the quantities project, which is not as flexible as pint and does not integrate very well with other packages. So, in this article we will work with the mass attribute and associate a pint unit with it.\nWhen we work with a gas, we need to decide which model will be used to describe it. The choice of model depends on the context. In a context where great accuracy is needed or near unusual conditions (very high pressure, for example), there are several models that can be used, the Van der Waals model probably being the best known. In our case, the simple ideal gas model, which is taught in high school physics and/or chemistry, is enough.\nThe ideal gas equation is:\n$$ PV = nRT $$where P, V, and T are pressure, volume, and temperature; n is the amount of substance; and R is the so-called ideal gas constant. Since the amount of substance of a given material is the mass of the substance m divided by its molar mass M, we can rewrite the equation as:\n$$ PV = \\frac{m}{M} RT $$Remembering that the density of a given material is its mass divided by its volume, we can rearrange the equation:\n$$ \\frac{m}{V} = \\frac{PM}{RT} \\therefore d = \\frac{PM}{RT} $$Thus, we see that there is a link between the molar mass of a substance and its density.\nThe SciPy library provides a large set of constants. At the beginning we imported the gas constant with the line\nfrom scipy.constants import gas_constant Let\u0026rsquo;s check the value:\ngas_constant 8.314462618 According to the library documentation, the value of the constant is in the International System of Units (SI). We can associate the unit with the value using the pint quantity constructor that we instantiated at the beginning of the article:\nQ_(gas_constant, \u0026#39;J/(mol K)\u0026#39;) 8.314462618 joule/(kelvin mole) Of course, you would not need to import SciPy just to get the value of this constant. You could simply put the value in a variable in the code and associate the unit. But I did it here in the article for two reasons. The first is informative: to show that SciPy, one of the most sensational scientific libraries, has this collection of constants. The second is to show the number of significant figures in the value that the library provides. I will write a bit more about that.\nIt is a (bad) widespread practice in Brazilian schools to round values without a more detailed discussion of the consequences. As a result, large errors end up appearing in calculations because of the roundings. You, who have read this article up to here, certainly have an interest in science, so keep one thing in mind: what must limit the precision of your calculations is the precision of your measurements. Every measurement has an associated uncertainty. But common constants in science, such as the ideal gas constant or the speed of light, usually go through several determination methods with high precision, and there is even the Committee on Data of the International Science Council (CODATA), which periodically publishes updated values of the main physical constants from which other constants can be derived.\nIf you clicked the link, notice the number of significant figures of the constants. It is highly likely that none of these constants will be what limits the precision of your calculations. It is no use, for example, to make an effort to get a pressure value that can be expressed with 4 significant figures, such as 1,200 Pa, and then round the magnitude of the gas constant to 8.31. You threw away all your experimental effort. Use physical constants with as many significant figures as are available. Each badly done rounding makes you lose a year of life (an exaggeration so that you remember the message).\nOh, and you may find the value shown for the gas constant strange. Maybe you thought \u0026ldquo;wait, wasn\u0026rsquo;t it 0.082?\u0026rdquo; Yes, in another system of units. The SciPy value is in the International System of Units (SI). In high school, and even in practical situations, we usually use the value best suited to working with the units atmosphere and liter. See the conversion:\nQ_(gas_constant, \u0026#39;J/(mol K)\u0026#39;).to(\u0026#39;atm l / (mol K)\u0026#39;) 0.08205736607944732 liter standard_atmosphere/(kelvin mole) Again, pay attention to the significant figures of the value.\nThe function that will support all the work # Well, let\u0026rsquo;s go back to our ideal gas. We can create a function that calculates the density of a gas. According to the equation we obtained above, we only need the pressure, the molar mass, and the temperature. But, with the help of chempy, we can get the molar mass from the gas formula. So, our function can receive a formula, the pressure, and the temperature. We can consider default values for the pressure and temperature, for ambient conditions, for example:\ndef gas_density(formula, pressure=Q_(\u0026#39;1 atm\u0026#39;), temperature=Q_(25, \u0026#39;degC\u0026#39;)): \u0026#34;\u0026#34;\u0026#34;Calculate the density of an ideal gas\u0026#34;\u0026#34;\u0026#34; R = Q_(gas_constant, \u0026#39;J/(mol K)\u0026#39;) M = chempy.Substance.from_formula(formula).mass M = Q_(M, \u0026#39;g/mol\u0026#39;) return (pressure * M/(R * temperature.to(\u0026#39;kelvin\u0026#39;))).to_base_units() The details of using the pint package can be read here.\nNow we can pass a formula and we will get the density for conditions of 1 atm and 25 °C.\ngas_density(\u0026#39;CO2\u0026#39;) 1.7988258254712144 kilogram/meter\u0026lt;sup\u0026gt;3\u0026lt;/sup\u0026gt; If we want other conditions, we just pass the pressure and temperature parameters. In any unit, in fact, since pint takes care of the conversion internally.\ngas_density(\u0026#39;CO2\u0026#39;, pressure=Q_(\u0026#39;1 * 10^4 Pa\u0026#39;), temperature=Q_(0, \u0026#39;degC\u0026#39;)) 0.1937787323145848 kilogram/meter\u0026lt;sup\u0026gt;3\u0026lt;/sup\u0026gt; Great, we can already calculate the density of a gas. But what about our atmosphere, which is a mixture of gases?\nWeb scraping with pandas # First we need a source that gives us the gases and their relative amounts. This Wikipedia link has a table with this information. Let\u0026rsquo;s assign this link to a variable:\nurl = \u0026#39;https://en.wikipedia.org/wiki/Atmosphere_of_Earth\u0026#39; Now let\u0026rsquo;s understand one of the imports we made at the beginning of the article, the pandas library. This library is widely used for data analysis and manipulation and is excellent for data that can be organized in tables. In the context of pandas, a set of tabular data is called a DataFrame.\nThe read_html method gives us a list of the tables at a URL as DataFrame objects:\nlen(pd.read_html(url)) 14 Note that 14 objects that can be handled as DataFrames were found at the link. You can go through each one to see which is the one you are interested in. In our case, it is right at the first position of the list, remembering that storage structures in Python, such as lists, start at index zero. So:\npd.read_html(url)[0] Gas Volume(A) Name Formula in ppmv(B) in % 0 Nitrogen N2 780840 78.084 1 Oxygen O2 209460 20.946 2 Argon Ar 9340 0.9340 3 Carbon dioxide(December, 2020)(C)[13] CO2 415.00 0.041500 4 Neon Ne 18.18 0.001818 5 Helium He 5.24 0.000524 6 Methane CH4 1.87 0.000187 7 Krypton Kr 1.14 0.000114 8 Not included in above dry atmosphere: Not included in above dry atmosphere: Not included in above dry atmosphere: Not included in above dry atmosphere: 9 Water vapor(D) H2O 0-30,000(D) 0-3%(E) 10 notes: (A) volume fraction is equal to mole fr... notes: (A) volume fraction is equal to mole fr... notes: (A) volume fraction is equal to mole fr... notes: (A) volume fraction is equal to mole fr... The notes section of the table was also recognized by pandas as part of the object. We are not interested in this part, so let\u0026rsquo;s remove it. Since this part starts at index 8 of the DataFrame (the index is the first column), let\u0026rsquo;s take a slice with iloc:\npd.read_html(url)[0].iloc[:8] Gas Volume(A) Name Formula in ppmv(B) in % 0 Nitrogen N2 780840 78.084 1 Oxygen O2 209460 20.946 2 Argon Ar 9340 0.9340 3 Carbon dioxide(December, 2020)(C)[13] CO2 415.00 0.041500 4 Neon Ne 18.18 0.001818 5 Helium He 5.24 0.000524 6 Methane CH4 1.87 0.000187 7 Krypton Kr 1.14 0.000114 This will be our DataFrame, so let\u0026rsquo;s assign it to a variable:\ndf = _ It is always interesting to take a look at how pandas recognized the data. For that we have the info and describe methods:\ndf.info() \u0026lt;class 'pandas.core.frame.DataFrame'\u0026gt; RangeIndex: 8 entries, 0 to 7 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 (Gas, Name) 8 non-null object 1 (Gas, Formula) 8 non-null object 2 (Volume(A), in ppmv(B)) 8 non-null object 3 (Volume(A), in %) 8 non-null object dtypes: object(4) memory usage: 384.0+ bytes df.describe() Gas Volume(A) Name Formula in ppmv(B) in % count 8 8 8 8 unique 8 8 8 8 top Nitrogen Ar 1.87 0.041500 freq 1 1 1 1 Note that pandas recognized all the columns as being of type object. This type is generic in pandas. Usually, when the algorithm does not recognize the type exactly, it classifies it as object. For all purposes, the data in these columns behave like strings. This is clear in the result of describe because, if the columns with numbers had been recognized as such, metrics such as mean, maximum value, and minimum value would appear. So, one of the things we will need to do is convert the column of volume in percentage to the correct type, float, so that we can use the values in our calculations.\nAnother aspect that these methods let us see more clearly is that there are subcolumns or, in language closer to that used in the pandas context, there are multi-indexes. Inside the Gas index there are Name and Formula, and there are also indexes inside Volume(A). Let\u0026rsquo;s see how to view the values of the Formula column:\ndf[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;] 0 N2 1 O2 2 Ar 3 CO2 4 Ne 5 He 6 CH4 7 Kr Name: Formula, dtype: object Another way would be using the notation of the loc method:\ndf.loc[:, (\u0026#39;Gas\u0026#39;, \u0026#39;Formula\u0026#39;)] 0 N2 1 O2 2 Ar 3 CO2 4 Ne 5 He 6 CH4 7 Kr Name: (Gas, Formula), dtype: object We can see only the data under Volume(A), which are our numerical data:\ndf[\u0026#39;Volume(A)\u0026#39;] in ppmv(B) in % 0 780840 78.084 1 209460 20.946 2 9340 0.9340 3 415.00 0.041500 4 18.18 0.001818 5 5.24 0.000524 6 1.87 0.000187 7 1.14 0.000114 Now let\u0026rsquo;s convert this data to the float type:\ndf[\u0026#39;Volume(A)\u0026#39;] = df[\u0026#39;Volume(A)\u0026#39;].astype(float) df Gas Volume(A) Name Formula in ppmv(B) in % 0 Nitrogen N2 780840.00 78.084000 1 Oxygen O2 209460.00 20.946000 2 Argon Ar 9340.00 0.934000 3 Carbon dioxide(December, 2020)(C)[13] CO2 415.00 0.041500 4 Neon Ne 18.18 0.001818 5 Helium He 5.24 0.000524 6 Methane CH4 1.87 0.000187 7 Krypton Kr 1.14 0.000114 Let\u0026rsquo;s now get ready to create a column for the density of each gas listed in the Formula subcolumn. Let\u0026rsquo;s see what the result would be of applying the gas_density function to each formula using Python\u0026rsquo;s map function:\nlist(map(gas_density, df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;])) [1.145045483304565 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 1.3078876766894934 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 1.6328363306579126 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 1.7988258254712144 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 0.8248259562876108 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 0.1636025323611701 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 0.6557422963038172 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;, 3.4251631830497584 \u0026lt;Unit('kilogram / meter ** 3')\u0026gt;] Note that the units appear, since our function uses pint. It would be interesting to have only the numerical value, using the magnitude attribute of a pint quantity, since it makes no sense for a unit to appear in every row of a table. It would also make our lives easier in future calculations to have only numbers. We can create an anonymous function for that and apply it with map to the previous result:\nlist( map(lambda quantity: quantity.magnitude, map(gas_density, df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;])) ) [1.145045483304565, Let\u0026rsquo;s store the previous result in a column of our DataFrame:\ndf[\u0026#39;density\u0026#39;] = _ df[\u0026#39;density\u0026#39;] 0 1.145045 Recalling our assumptions, these values are the density of each gas separately, considering an ideal gas model, a pressure of 1 atm, and a temperature of 25 °C.\nDensity and mean molar mass of air # As explained in the Wikipedia link from which we took the table, the percentage values shown consider dry air, since the moisture content varies a lot even in a given location. So, when we talk about the density of air, we usually mean the density of dry air. This density is nothing more than the weighted average of the densities of each constituent of air. The weights are the percentages of each constituent. Thus:\ndry_air_density = sum([gas_density * gas_percentage for gas_density, gas_percentage in zip(df[\u0026#39;density\u0026#39;], df[\u0026#39;Volume(A)\u0026#39;][\u0026#39;in %\u0026#39;])]) / sum(df[\u0026#39;Volume(A)\u0026#39;][\u0026#39;in %\u0026#39;]) dry_air_density 1.1839692449104982 This is the calculated value of the density of dry air. Considering the number of significant figures of the data presented and adding the proper unit, it is more appropriate to represent the result as:\nprint(f\u0026#34;{Q_(dry_air_density, \u0026#39;kg/m^3\u0026#39;):.2~P}\u0026#34;) 1.2 kg/m³ Following the same line of reasoning, we can calculate the mean molar mass:\ndf[\u0026#39;molar mass\u0026#39;] = [chempy.Substance.from_formula(formula).mass for formula in df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;]] mean_molar_mass_dry_air = sum([gas_molar_mass * gas_percentage for gas_molar_mass, gas_percentage in zip(df[\u0026#39;molar mass\u0026#39;], df[\u0026#39;Volume(A)\u0026#39;][\u0026#39;in %\u0026#39;])]) / sum(df[\u0026#39;Volume(A)\u0026#39;][\u0026#39;in %\u0026#39;]) print(f\u0026#34;{Q_(mean_molar_mass_dry_air, \u0026#39;g/mol\u0026#39;):.4~P}\u0026#34;) 28.97 g/mol Plots! I want images # Now let\u0026rsquo;s use all the power of Matplotlib to create plots that make our analysis easier to visualize and even expand our understanding further.\npandas has a plot method that is built on top of Matplotlib\u0026rsquo;s plot. We can, for example, easily create a bar plot with the densities of each gas in our DataFrame:\ndf.plot.bar(x=(\u0026#39;Gas\u0026#39;, \u0026#39;Formula\u0026#39;), y=\u0026#39;density\u0026#39;) Matplotlib\u0026rsquo;s default settings are good, but there is a lot to improve in this plot. After all, the unit is missing on the vertical axis, the axis titles could be better, and the numbers in the chemical formulas should be subscripts. Let\u0026rsquo;s fix these issues.\nFor the chemical formulas, we can create a function that, with the help of regular expressions, returns the LaTeX code needed to display the formulas correctly. That is because Matplotlib interacts very well with LaTeX, which is great for those of us in science who use it a lot in our work.\nimport re def format_formula(formula): \u0026#34;\u0026#34;\u0026#34; Return LaTeX code to display chemical formulas correctly\u0026#34;\u0026#34;\u0026#34; label_formula = re.sub(\u0026#34;([0-9])\u0026#34;, r\u0026#34;_\\1\u0026#34;, formula) return \u0026#39;\u0026#39;.join([\u0026#39;$\\mathregular{\u0026#39;, label_formula, \u0026#39;}$\u0026#39;]) Basically, the function adds a _ before any digit so that the LaTeX code recognizes the digit as a subscript. To learn more about regular expressions, read this excellent article here on the site.\nMatplotlib\u0026rsquo;s default color scheme is good and presentable, but I particularly like plots with a dark background. So, I will use one of the ready-made styles available in the library:\nplt.style.use(\u0026#39;seaborn-whitegrid\u0026#39;) Now let\u0026rsquo;s create a figure, associate a set of axes with it, and add the information we wanted: units, axis names, correct chemical formulas, and, while we\u0026rsquo;re at it, a line indicating the density of dry air that we calculated earlier:\nfig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8,6), constrained_layout=True) # Data ax.bar(df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;], df[\u0026#39;density\u0026#39;], color=\u0026#39;blue\u0026#39;, label=\u0026#39;Gas density\u0026#39;) ax.axhline(dry_air_density, color=\u0026#39;red\u0026#39;, label=\u0026#39;Dry air density\u0026#39;) # Correct chemical formulas ax.set_xticklabels(list(map(format_formula, df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;]))) # Labels ax.set_xlabel(\u0026#39;Compound\u0026#39;, size=14) ax.set_ylabel(\u0026#39;Density / kg/m³\u0026#39;, size=14) ax.set_title(\u0026#39;Density of the gases that make up dry air - 1 atm and 25 °C\u0026#39;, size=16) ax.legend(fontsize=12) # Tick label font size ax.tick_params(labelsize=14) plt.show() This plot is interesting, but it does not give the reader enough information to understand why the density of dry air is much closer to that of nitrogen gas than to that of krypton gas. Of course, the more attentive will deduce that this probably happens because there is very little krypton in the atmosphere, but it would be useful for this data to be explicit in the plot.\nIn addition, if the audience is more academic, for example in a class, the plot could also include some indication that the density of a given gas is proportional to its molar mass. That way, more discussions could be had about gas models and even about other consequences, depending on the instructor\u0026rsquo;s goal.\nSo, now let\u0026rsquo;s make a bolder plot to end this article with a flourish. We can create a color intensity scale to indicate the concentration of the gas in the atmosphere. For example, a pale yellow would indicate low concentration and a strong red, high concentration. A bar plot limits us a bit, so let\u0026rsquo;s switch to a scatter plot. In this type of plot, the size of the point, or rather the circle, can be associated with a quantity. So, we will associate it with the molar mass value to get the additional information we want.\nTo create the color scale, we will need two Matplotlib modules that we have not imported yet:\nimport matplotlib.colors as mcolors import matplotlib.cm as cm So, let\u0026rsquo;s create a figure with a set of axes and make the necessary adjustments to our plot. We will use Matplotlib\u0026rsquo;s YlOrRd colormap and normalize the colors based on the volume percentage values. That way, the gas with the lowest percentage will have the palest yellow and the one with the highest, a dark red. With this data, we will use Matplotlib methods to build a color scale. Using chempy, we will get the molar mass of each formula in the DataFrame and associate the values with the area of each circle. The following code does all these steps. I recommend that you run the code on your machine and interact with it, modifying each part and seeing its effects. You can learn a lot of Matplotlib here.\nfig, ax = plt.subplots(nrows=1, ncols=1, figsize=(10,6), constrained_layout=True) # circle size based on molar mass base_radius = 15 # change for smaller or larger circles circle_areas = [chempy.Substance.from_formula(i).mass * base_radius for i in df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;]] # circle colors based on volume percentage values circle_colors = df[\u0026#39;Volume(A)\u0026#39;][\u0026#39;in %\u0026#39;] normalize = mcolors.Normalize(vmin=min(circle_colors), vmax=max(circle_colors)) colormap = cm.rainbow # data sc = ax.scatter(df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;], df[\u0026#39;density\u0026#39;], label=\u0026#39;Gas density - Circle proportional to molar mass\u0026#39;, s=circle_areas, c=circle_colors, cmap=colormap) ax.axhline(dry_air_density, color=\u0026#39;red\u0026#39;, label=\u0026#39;Dry air density\u0026#39;) # color scale scalar_mappable = cm.ScalarMappable(norm=normalize, cmap=colormap) scalar_mappable.set_array(circle_colors) colorbar = fig.colorbar(scalar_mappable) # Labels ax.set_xticklabels(list(map(format_formula, df[\u0026#39;Gas\u0026#39;][\u0026#39;Formula\u0026#39;]))) legend = ax.legend(scatterpoints=3, fontsize=12) legend.legendHandles[1]._sizes = [30] ax.set_xlabel(\u0026#39;Compound\u0026#39;, size=18) ax.set_ylabel(\u0026#39;Density / kg/m³\u0026#39;, size=18) colorbar.ax.set_ylabel(\u0026#39;Dry air composition, in % by volume\u0026#39;, rotation=270, labelpad=20, size=15) fig.suptitle(\u0026#39;Density of the gases that make up dry air - 1 atm and 25 °C\u0026#39;, size=20) # Tick label font size colorbar.ax.tick_params(labelsize=14) ax.tick_params(labelsize=14) plt.show() Now we have a plot very rich in information: the density of each gas in isolation, the density of dry air, the relation between density and molar mass, and the percentage composition of air. All of this in just two dimensions, which is excellent for static visualization. Three-dimensional plots are great when the person can interact, rotating them and zooming in. But in printed or online material that is static, they can be a bit confusing. So, putting more information into a two-dimensional plot is a better choice.\nConclusion # So, did you like it? Here is a short list of the tools we used:\nPint ChemPy SciPy pandas Web scraping with pandas Regular expressions Matplotlib ready-made styles color scale scatter plot advanced plot customization Standard Python map anonymous functions (lambda) list comprehensions If you think this article helped you, share it on your social networks. It helps this project, which is only now starting to become better known, a lot. It is a simple request, but it helps a lot.\nSee you next time!\n","date":"2020-12-05","externalUrl":null,"permalink":"/2020/12/studying-atmosphere-python/","section":"Blog","summary":"Colorless, made of invisible constituents, and odorless (we hope!). We often forget that we are surrounded by the gases that make up our atmosphere. In this article, we will see how to easily get a list of the main constituents of air with Python and web scraping. We will also do some calculations, such as density and mean molar mass, and, of course, present everything in spectacular plots.","title":"Studying Our Atmosphere with Python","type":"posts"},{"content":"In this video (in Portuguese) I show how to manage issues and pull requests on GitHub, both through the website interface and from the terminal of your local repository.\nLink to the project repository shown in the video.\n","date":"2020-12-02","externalUrl":null,"permalink":"/2020/12/managing-issues-pull-requests-github/","section":"Blog","summary":"We will see how to manage issues and pull requests on GitHub, both through the website interface and from the terminal of your local repository.","title":"Managing Issues and Pull Requests on GitHub","type":"posts"},{"content":"","date":"2020-11-25","externalUrl":null,"permalink":"/tags/cli/","section":"tags","summary":"","title":"CLI","type":"tags"},{"content":"In this video (in Portuguese) I show how to create a very simple command-line interface (CLI) in Python and, along the way, how to use branches in Git.\nIt is very common for beginner programming materials to tell students to write programs that ask the user for some information, process it, and display the result on screen. For simplicity, such programs are usually command-line programs that run in the terminal. However, when you deal with input straight from the user, some care is needed. In this video I show a few ways to handle wrong user input and, along the way, I introduce a few more Git concepts, such as the use of branches.\nThe video starts from the program created earlier here, where I show how to compute the factorial of a number in Python with just one line.\nThe links to the sites and other materials mentioned in the video are right after the video player:\nRepository with the code.\nRepository of the Rich project for better formatting in the terminal.\nRepository of the Oh My Zsh project, a configuration framework for Zsh.\nIn the video, I show the following generic structure for a very simple Python CLI:\nif __name__ == \u0026#39;__main__\u0026#39;: while True: try: variable = input(\u0026#39;Text asking for something: \u0026#39;) # code that does the initial processing of what was received in the variable except ValueError: print(\u0026#39;Error message for the user\u0026#39;) # create as many except clauses as needed for the cases that may occur in your code except KeyboardInterrupt: print(\u0026#39; Execution ended by the user.\u0026#39;) break else: print(\u0026#39;Message for the user with the result\u0026#39;) # Edit the message above to show whatever you want to the user. # Create more prints if needed. ","date":"2020-11-25","externalUrl":null,"permalink":"/2020/11/creating-python-cli-git-branches/","section":"Blog","summary":"Let’s create a very simple command-line interface (CLI) in Python and, along the way, I show how to use branches in Git.","title":"Creating a CLI with Python and Using Branches in Git","type":"posts"},{"content":"In this video (in Portuguese) I show how to structure a small Python project with the help of a very simple exercise. I present the steps to set up whatever editor or IDE you use, how to work with version control, in this case Git, and how to develop with tests (TDD, Test Driven Development).\nWhen we are starting to learn Python, or any other programming language, it is very common to use a book, a reference website, or some other kind of material. And these materials usually have exercises to practice. However, there is rarely any instruction on how to structure your study environment in terms of code organization or configuring your editor or IDE.\nHere, I use a very simple exercise, computing the factorial of a number, taken from the QuantEcon website, to show how to be more efficient in your studies and, on top of that, learn important concepts such as versioning with Git and test-driven development.\nThe links to websites and other materials mentioned in the video are right after the video player:\nSummary of the steps:\nOrganize a folder for the project The .gitignore file Initialize a Git repository Select the virtual environment (or create one for the project) select the interpreter install, if necessary, packages for the project Write tests and work on making them pass Make commits at relevant points of the project Work with branches when relevant, always keeping a presentable and working version on master Repository with the code.\nBasic Python course on the QuantEcon website.\nArticle here on the site about the Anaconda project.\nArticle here on the site about virtual environments.\nArticle here on the site about IPython.\n","date":"2020-11-02","externalUrl":null,"permalink":"/2020/11/efficient-python-tdd-git-factorial/","section":"Blog","summary":"Let’s see how to structure a small Python project with the help of a very simple exercise. I will show the steps to set up whatever editor or IDE you use, how to work with version control (in this case Git), and how to develop with tests (TDD, Test Driven Development).","title":"Efficient Python: TDD tests, Git, and factorial solved in one line","type":"posts"},{"content":"","date":"2020-11-02","externalUrl":null,"permalink":"/tags/tdd/","section":"tags","summary":"","title":"TDD","type":"tags"},{"content":"","date":"2020-11-02","externalUrl":null,"permalink":"/tags/testes/","section":"tags","summary":"","title":"testes","type":"tags"},{"content":"","date":"2020-11-02","externalUrl":null,"permalink":"/tags/testing/","section":"tags","summary":"","title":"testing","type":"tags"},{"content":"","date":"2020-09-27","externalUrl":null,"permalink":"/tags/ipython/","section":"tags","summary":"","title":"IPython","type":"tags"},{"content":"In this video (in Portuguese) we will see the advantages of using an interactive Python interpreter over the language\u0026rsquo;s standard interpreter. IPython can help you study the language and produce Python-related content.\nTo watch the video, click the player below. The links mentioned in the video come right after it.\nLink to the IPython project website.\nLink to the documentation of the various magic commands in IPython.\nExplanation of the different times shown by the %time command.\nLink to the file exported by IPython with all the content of the tutorial.\nI have already written and made a video about the Anaconda distribution.\nSee this article and this one as well, where I use Jupyter Notebooks.\n","date":"2020-09-27","externalUrl":null,"permalink":"/2020/09/learn-python-interactive-interpreter/","section":"Blog","summary":"We will see the advantages of using an interactive Python interpreter over the language’s standard interpreter. IPython can help you study the language and produce Python-related content.","title":"Learn Python with an interactive interpreter","type":"posts"},{"content":"Regular expressions (also called regex, short for regular expressions) are a language that detects patterns. You write a pattern that will be applied to strings. It is an efficient way to find patterns and extract information from text. In a pattern, each symbol represents a type of information and, in this article, we will see the main symbols and their meanings with the help of several examples.\nPython has the re module for regular expressions. The documentation is very detailed and there is also this text with a more step-by-step introduction to the module.\nThe module has three methods that are widely used and that will be our focus here: match, search and findall.\nMatch # match looks for the given pattern from the start of the string. The signature is re.match(pattern, string, flags=0). The return value is None when the pattern is not found. When it is found, it returns a Match object, which we will discuss throughout the article.\nLet\u0026rsquo;s start by looking for the pattern \u0026ldquo;Fran\u0026rdquo; in the string \u0026ldquo;Francisco Bustamante\u0026rdquo;, the author of the site:\n\u0026gt;\u0026gt;\u0026gt; import re \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;Fran\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(0, 4), match=\u0026#39;Fran\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;Fran\u0026#39;, \u0026#39;Bustamante, Francisco\u0026#39;) # note that the last one returns None because it does not match at the start span=(0, 4) means that the pattern was found between indices 0 and 4. Remember that counting in Python sequences starts at zero and that the last index is exclusive.\nSearch # search looks for the given pattern along the string. The signature is re.search(pattern, string, flags=0). The return value is None when the pattern is not found. When it is found, it returns a Match object, which we will discuss throughout the article. It is important to stress that it returns the first location where the pattern was found.\n\u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;an\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(2, 4), match=\u0026#39;an\u0026#39;\u0026gt; The pattern an appears in the string more than once, but search returns in span only the indices of the first location.\nFindall # findall looks for the given pattern along the string. The signature is re.findall(pattern, string, flags=0). The return value is an empty list when the pattern is not found. When it is found, it returns a list with each occurrence. It is important to stress that it returns all the occurrences found.\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;an\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) [\u0026#39;an\u0026#39;, \u0026#39;an\u0026#39;] \u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;an\u0026#39;, \u0026#39;Chico\u0026#39;) [] The power of findall will be better explored with some tools that we will learn throughout the article.\nFlags # The signature of the methods presented has a flags argument. With flags we can modify some aspects of how regular expressions work. See the difference in behavior in the following two examples:\n\u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;fran\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;fran\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;, re.IGNORECASE) \u0026lt;re.Match object; span=(0, 4), match=\u0026#39;Fran\u0026#39;\u0026gt; In the first case, the return value was None, so the interpreter just showed an empty line. That is because the pattern was given with a lowercase first letter and, in the string, it is uppercase. The re.IGNORECASE flag, as the name suggests, disregards the difference between uppercase and lowercase.\nBelow is a table with the main flags of the re module and their meanings. The letters after each name are the abbreviations that can be used in place of the full name.\nFlag Meaning ASCII, A Treats escape sequences such as \\w, \\b, \\s and \\d as matching only ASCII characters DOTALL, S Makes the . metacharacter match any character, including newlines IGNORECASE, I Matches without distinguishing uppercase from lowercase LOCALE, L Matches taking the locale into account MULTILINE, M Multiline matching, affecting ^ and $ VERBOSE, X (for \u0026rsquo;extended') Enables verbose regular expressions, which can be organized in a clearer and more understandable way Some new words appeared in this table. Don\u0026rsquo;t worry, they will be explained throughout the article. One of them is metacharacter, our next topic.\nThe power of regular expressions comes from metacharacters, characters that represent a set of specific characters, general patterns.\nThe dot metacharacter (.) # The . metacharacter represents any character, except a line break. Let\u0026rsquo;s use it with the match method seen earlier:\n\u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;F\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39;42\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;4\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39; Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39; \u0026#39;\u0026gt; Note in the last example that there was a space at the start of the string and that space was recognized by the metacharacter.\nBy definition, the . metacharacter should match control characters except \\n, which indicates a line break. Let\u0026rsquo;s see:\n\u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39;\\t\\t\u0026#39;) # \\t represents TAB \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;\\t\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39;\\n\u0026#39;) \u0026gt;\u0026gt;\u0026gt; print(re.match(\u0026#39;.\u0026#39;, \u0026#39;\\n\u0026#39;)) None This behavior of ignoring \\n can be modified by one of the flags we saw earlier, DOTALL:\n\u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;.\u0026#39;, \u0026#39;\\n\u0026#39;, re.DOTALL) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;\\n\u0026#39;\u0026gt; We saw earlier that search looks for the pattern along the string and returns the first position where it finds it. Let\u0026rsquo;s see the behavior with .:\n\u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;.\u0026#39;, \u0026#39; Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39; \u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;.\u0026#39;, \u0026#39;Francisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;F\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;.\u0026#39;, \u0026#39;\\nFrancisco Bustamante\u0026#39;) \u0026lt;re.Match object; span=(1, 2), match=\u0026#39;F\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;.\u0026#39;, \u0026#39;\\nFrancisco Bustamante\u0026#39;, re.DOTALL) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;\\n\u0026#39;\u0026gt; The first two examples return the first position, as expected. In the third example, the control character \\n is ignored, returning the first character right after it, the letter F. This behavior is modified by the DOTALL flag in the last example.\nWe saw earlier that findall looks for the given pattern along the string, returning a list with all the occurrences. Let\u0026rsquo;s combine it with .:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;.\u0026#39;, \u0026#39;Ciência\\nProgramada\u0026#39;) [\u0026#39;C\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;ê\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;m\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] The control character \\n was ignored and all the others were returned as a list.\nAnchors # The ^ symbol represents the start of a string and the $ symbol represents the end of a string. Let\u0026rsquo;s test with the findall method:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;^.\u0026#39;, \u0026#39;Ciência\\nProgramada\\nPython\u0026#39;) [\u0026#39;C\u0026#39;] \u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;^.\u0026#39;, \u0026#39;Ciência\\nProgramada\\nPython\u0026#39;, re.MULTILINE) [\u0026#39;C\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;P\u0026#39;] The pattern ^. means looking for any character other than a line break at the start of the string. So, in the first case, it returns the letter C. But note that the given string has line breaks. So we can pass the re.MULTILINE flag so that the pattern is searched for in each line. That is why the second example returns a list with the first letter of the string and also the first letter after each \\n control character.\nWe can apply the same logic to the pattern .$, which searches at the end of the string:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;.$\u0026#39;, \u0026#39;Ciência\\nProgramada\\nPython\u0026#39;) [\u0026#39;n\u0026#39;] \u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;.$\u0026#39;, \u0026#39;Ciência\\nProgramada\\nPython\u0026#39;, re.MULTILINE) [\u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;n\u0026#39;] Some edge cases we can analyze are when we have a string with only one character, or an empty one, or one with a line break:\n\u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;^.$\u0026#39;, \u0026#39;a\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;a\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(\u0026#39;^$\u0026#39;, \u0026#39;\u0026#39;) # the start equals the end, empty string \u0026lt;re.Match object; span=(0, 0), match=\u0026#39;\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;^$\u0026#39;, \u0026#39;\\n\u0026#39;, re.MULTILINE) [\u0026#39;\u0026#39;, \u0026#39;\u0026#39;] The . metacharacter is very broad; we usually want to be a bit more specific.\nCharacter classes # When the pattern has square brackets, they declare a character class. Each character between the brackets will be searched for in the text of the string. Let\u0026rsquo;s look for lowercase vowels in the string Ciência Programada:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;[aeiou]\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;i\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;] Note that ê was not put in the list, after all it is different from e.\nThe ^ symbol inside a character class means negation. So, if we are looking for everything except lowercase vowels:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;[^aeiou]\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;C\u0026#39;, \u0026#39;ê\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39; \u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;m\u0026#39;, \u0026#39;d\u0026#39;] It is also possible to define ranges of values. Searching from \u0026ldquo;a\u0026rdquo; to \u0026ldquo;f\u0026rdquo;:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;[a-f]\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;c\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] We can define more than one range. Searching from \u0026ldquo;a\u0026rdquo; to \u0026ldquo;f\u0026rdquo; and from \u0026ldquo;A\u0026rdquo; to \u0026ldquo;Z\u0026rdquo;:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;[a-fA-Z]\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;C\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] A very common pattern is to search for all letters and digits and for \u0026ldquo;_\u0026rdquo;, since they tend to be the characters most accepted in online form fields such as e-mail, for example:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;[a-zA-Z0-9_]\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;C\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;m\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] It is such a special sequence that there is a shortcut, \\w:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;\\w\u0026#39;, \u0026#39;Ciência Programada\u0026#39;) [\u0026#39;C\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;ê\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;m\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] Note two small differences between this result and the previous example. First, square brackets are not needed to use \\w. Second, the character ê appears in the result. That is, here Unicode characters are recognized. If you really want only the equivalent of the class [a-zA-Z0-9_], use the re.ASCII flag:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;\\w\u0026#39;, \u0026#39;Ciência Programada\u0026#39;, re.ASCII) [\u0026#39;C\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;r\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;m\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;a\u0026#39;] The main special sequences defined by default are:\n\\d equivalent to any Unicode digit, which includes [0-9] \\D equivalent to the negation of \\d \\s equivalent to Unicode whitespace characters, which includes [ \\t\\n\\r\\f\\v] \\S equivalent to the negation of \\s \\w equivalent to characters that can be used in text in general, which includes [a-zA-Z0-9_] \\W equivalent to the negation of \\w When the re.ASCII flag is used, each case above is restricted to the bracket representation shown. In Unicode they are broader.\nRaw strings # All special sequences use the \\ symbol. This is problematic in Python, because strings accept control characters in the language. Literal strings evaluate the character after the backslash to check whether it is a control character or not. Let\u0026rsquo;s see an example:\n\u0026gt;\u0026gt;\u0026gt; print(\u0026#39;1\\n2\u0026#39;) 1 2 When we want to indicate that it should not be treated as a control character, we use another backslash to signal escape:\n\u0026gt;\u0026gt;\u0026gt; print(\u0026#39;1\\\\n2\u0026#39;) # escape of the second backslash 1\\n2 This behavior related to backslashes can be very problematic in some contexts. For example, in science LaTeX is widely used to produce documents. And LaTeX environments are delimited with commands that use \\. For example, \\begin{equation}...\\end{equation} delimits an environment for a mathematical equation. The \\section command indicates the start of a section in a document. Let\u0026rsquo;s see how the Python interpreter recognizes the command followed by a line break:\n\u0026gt;\u0026gt;\u0026gt; \u0026#39;\\section\\n\u0026#39; # \\s has no meaning in Python \u0026#39;\\\\section\\n\u0026#39; \u0026gt;\u0026gt;\u0026gt; text = \u0026#39;\\section\\n\u0026#39; # LaTeX \u0026gt;\u0026gt;\u0026gt; print(text) \\section Note that, since \\s has no meaning for the Python interpreter, it automatically adds a backslash to signal that this character should be ignored. When we use print, the text of the command appears normally, followed by a blank line.\nWe can verify that control characters are counted as a single character by checking the length of the string stored in the variable text:\n\u0026gt;\u0026gt;\u0026gt; len(text) 9 There are 8 characters in \u0026ldquo;\\section\u0026rdquo; and the ninth is the control character \\n.\nIn theory, there should be a match when searching for \\\\section in the string stored in text:\n\u0026gt;\u0026gt;\u0026gt; print(re.match(\u0026#39;\\\\section\u0026#39;, text)) # \\s is a special sequence in regex None But, as we saw in the previous section, \\s has a meaning in the context of the re module. So we need to indicate that the backslashes must be escaped:\n\u0026gt;\u0026gt;\u0026gt; print(re.match(\u0026#39;\\\\\\\\section\u0026#39;, text)) \u0026lt;re.Match object; span=(0, 8), match=\u0026#39;\\\\section\u0026#39;\u0026gt; Welcome to what we call the backslash plague! But calm down, there is a way to avoid all those backslashes. In Python, there are what we call raw strings. In this type of string, denoted by an r before the quotes, backslashes are not treated in any special way. Notice the difference:\n\u0026gt;\u0026gt;\u0026gt; len(\u0026#39;\\n\u0026#39;) 1 \u0026gt;\u0026gt;\u0026gt; len(r\u0026#39;\\n\u0026#39;) 2 In the raw string, we have two characters, \u0026ldquo;\u0026quot; and \u0026ldquo;n\u0026rdquo;, whereas in the normal (literal) string there is only one character, which represents a line break.\nBack to our example, we just need to use a raw string in the pattern:\n\u0026gt;\u0026gt;\u0026gt; print(re.match(r\u0026#39;\\\\section\u0026#39;, text)) # raw string, meaning there is no control character \u0026lt;re.Match object; span=(0, 8), match=\u0026#39;\\\\section\u0026#39;\u0026gt; So, a very important tip: use raw strings whenever there is a backslash that should not be interpreted as a special sequence.\nUsing the pipe # The pipe | means or, indicating alternatives in the regular expression. See examples:\n\u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;a|b\u0026#39;, \u0026#39;abc\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;a\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;a|b\u0026#39;, \u0026#39;bcd\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;b\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.search(\u0026#39;a|b\u0026#39;, \u0026#39;cde\u0026#39;) Note in the first example that, even though \u0026ldquo;b\u0026rdquo; exists in the string, since \u0026ldquo;a\u0026rdquo; was found first only the position of \u0026ldquo;a\u0026rdquo; was returned. With findall, both are returned:\n\u0026gt;\u0026gt;\u0026gt; re.findall(\u0026#39;b|a\u0026#39;, \u0026#39;abc\u0026#39;) [\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;] Repetitions # Searching for repetitions is one of the most common reasons for using regular expressions. Let\u0026rsquo;s look at the various possible forms.\nSpecific quantities # Let\u0026rsquo;s check the behavior of searching for the pattern \\d{4}, which looks for any digit four times.\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{4}\u0026#39;, \u0026#39;1234\u0026#39;) # string with 4 digits \u0026lt;re.Match object; span=(0, 4), match=\u0026#39;1234\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{4}\u0026#39;, \u0026#39;123\u0026#39;) # string with 3 digits \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{4}\u0026#39;, \u0026#39;12345\u0026#39;) # string with 5 digits \u0026lt;re.Match object; span=(0, 4), match=\u0026#39;1234\u0026#39;\u0026gt; In the examples, it is clear that, when there are fewer than four digits, the return value is None. In the other cases, it returns the first four digits. The same happens with the search method:\n\u0026gt;\u0026gt;\u0026gt; re.search(r\u0026#39;\\d{4}\u0026#39;, \u0026#39;abc123def12345\u0026#39;) \u0026lt;re.Match object; span=(9, 13), match=\u0026#39;1234\u0026#39;\u0026gt; The first segment 123 was ignored because it has fewer than 4 digits. The second segment, which has 5 digits, was considered and returned the first four digits.\nMinimum and maximum quantity # Using a comma indicates that the value being searched for is the minimum. That is, if there are more digits the behavior will be greedy and will return the digits beyond the minimum. If there are fewer digits than the minimum, the return value will be None.\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,}\u0026#39;, \u0026#39;12\u0026#39;) \u0026lt;re.Match object; span=(0, 2), match=\u0026#39;12\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,}\u0026#39;, \u0026#39;12345\u0026#39;) # greedy \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,}\u0026#39;, \u0026#39;1\u0026#39;) Using the ? symbol after the braces turns the behavior into lazy, so this symbol is a repetition modifier.\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,}?\u0026#39;, \u0026#39;12345\u0026#39;) # lazy, minimum possible \u0026lt;re.Match object; span=(0, 2), match=\u0026#39;12\u0026#39;\u0026gt; A value after the comma indicates the maximum. The previous explanations remain valid.\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,4}\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 4), match=\u0026#39;1234\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,4}\u0026#39;, \u0026#39;123\u0026#39;) \u0026lt;re.Match object; span=(0, 3), match=\u0026#39;123\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,4}\u0026#39;, \u0026#39;12\u0026#39;) \u0026lt;re.Match object; span=(0, 2), match=\u0026#39;12\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,4}\u0026#39;, \u0026#39;1\u0026#39;) \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{2,4}?\u0026#39;, \u0026#39;12345\u0026#39;) # from greedy to lazy \u0026lt;re.Match object; span=(0, 2), match=\u0026#39;12\u0026#39;\u0026gt; 0 or 1 occurrence, optional element # Searching for an optional element in the string is, in fact, a special case of minimum and maximum quantity, with a minimum of zero and a maximum of one.\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{0,1}\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;1\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{,1}\u0026#39;, \u0026#39;12345\u0026#39;) # 0 can be omitted \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;1\u0026#39;\u0026gt; The ? symbol after a regular expression has the same effect as {,1}. So:\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d?\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;1\u0026#39;\u0026gt; But we have already seen that the same symbol turns a greedy search into a lazy one. So the following expression returns an empty string, because the minimum is no (0) occurrences:\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d??\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 0), match=\u0026#39;\u0026#39;\u0026gt; Let\u0026rsquo;s go step by step. The first question mark is a repetition modifier of the regular expression immediately before it, looking for 0 or 1 occurrence of \\d. The second question mark is a modifier of the repetition operator, turning it lazy.\n0 or more times # Another special case of minimum and maximum. It also has a special symbol, *. Look at the examples:\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{0,}\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{,}\u0026#39;, \u0026#39;12345\u0026#39;) # 0 can be omitted \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d*\u0026#39;, \u0026#39;12345\u0026#39;) # symbol \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d*?\u0026#39;, \u0026#39;12345\u0026#39;) # lazy \u0026lt;re.Match object; span=(0, 0), match=\u0026#39;\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d*\u0026#39;, \u0026#39;abc\u0026#39;) \u0026lt;re.Match object; span=(0, 0), match=\u0026#39;\u0026#39;\u0026gt; In the last example, the match is an empty string, because the minimum is no occurrences.\n1 or more times # Another special case of minimum and maximum. It also has a special symbol, +. Look at the examples:\n\u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d{1,}\u0026#39;, \u0026#39;12345\u0026#39;) \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d+\u0026#39;, \u0026#39;12345\u0026#39;) # + requires at least one occurrence, being greedy \u0026lt;re.Match object; span=(0, 5), match=\u0026#39;12345\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d+?\u0026#39;, \u0026#39;12345\u0026#39;) # turns it lazy \u0026lt;re.Match object; span=(0, 1), match=\u0026#39;1\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; re.match(r\u0026#39;\\d+\u0026#39;, \u0026#39;abc\u0026#39;) In the last example, it returns None, because the minimum is one occurrence and there are no digits in the string.\nUnderstanding the importance of controlling repetitions # In case everything has seemed too abstract so far, let\u0026rsquo;s start putting in some situations closer to the real world. Consider the string below, from which we would like to extract all the attribute values.\n\u0026gt;\u0026gt;\u0026gt; text = \u0026#39;name=\u0026#34;Francisco\u0026#34; site=\u0026#34;Ciência Programada\u0026#34;\u0026#39; Since the values are between double quotes, at first we might imagine the pattern r'\u0026quot;.+\u0026quot;', since . would grab the characters, with the + modifier indicating one or more times. But look at the result:\n\u0026gt;\u0026gt;\u0026gt; re.findall(r\u0026#39;\u0026#34;.+\u0026#34;\u0026#39;, text) [\u0026#39;\u0026#34;Francisco\u0026#34; site=\u0026#34;Ciência Programada\u0026#34;\u0026#39;] It did not work because we matched anything between quotes with at least one occurrence. The search extends from the first double quote to the last. In fact, we want what is between each pair of quotes. So we must turn the search lazy:\n\u0026gt;\u0026gt;\u0026gt; re.findall(r\u0026#39;\u0026#34;.+?\u0026#34;\u0026#39;, text) [\u0026#39;\u0026#34;Francisco\u0026#34;\u0026#39;, \u0026#39;\u0026#34;Ciência Programada\u0026#34;\u0026#39;] Now we have what we wanted. But it is still not a good way to get the values. Look at what would happen if empty fields were passed:\n\u0026gt;\u0026gt;\u0026gt; text = \u0026#39;name=\u0026#34;\u0026#34; site=\u0026#34;\u0026#34;\u0026#39; \u0026gt;\u0026gt;\u0026gt; re.findall(r\u0026#39;\u0026#34;.+?\u0026#34;\u0026#39;, text) [\u0026#39;\u0026#34;\u0026#34; site=\u0026#34;\u0026#39;] It returns the wrong result because + requires at least one occurrence. In fact, we want 0 or more occurrences, which we can solve using *:\n\u0026gt;\u0026gt;\u0026gt; re.findall(r\u0026#39;\u0026#34;.*?\u0026#34;\u0026#39;, text) [\u0026#39;\u0026#34;\u0026#34;\u0026#39;, \u0026#39;\u0026#34;\u0026#34;\u0026#39;] Understanding the Match Object # In several examples we saw that the result is a Match object. Let\u0026rsquo;s understand this object a bit better. There are four important methods for the level of this article:\n\u0026gt;\u0026gt;\u0026gt; m = re.match(r\u0026#39;\\d+\u0026#39;, \u0026#39;12345\u0026#39;) \u0026gt;\u0026gt;\u0026gt; type(m) re.Match \u0026gt;\u0026gt;\u0026gt; m.group() # returns the string that was matched \u0026#39;12345\u0026#39; \u0026gt;\u0026gt;\u0026gt; m.start() # start position of the match 0 \u0026gt;\u0026gt;\u0026gt; m.end() # end position of the match 5 \u0026gt;\u0026gt;\u0026gt; m.span() # tuple with the start and end position of the match (0, 5) The group method will be important for the next section.\nCapture groups # Sometimes the string being analyzed has several fields from which we want to extract data. We can then define capture groups. Groups are delimited by parentheses and we can repeat the content of a group with a repetition qualifier, such as *, +, ?, or {m,n}. For example, (ab)* will match zero or more repetitions of ab.\nConsider the HTML tag below, from which we want to extract the name (input) and the values of type, id and name. We can create a pattern, the pattern variable in the code, following the format of the string. In each group we want to grab one or more occurrences of characters, lazily, to avoid the problems seen in the section on controlling repetitions:\n\u0026gt;\u0026gt;\u0026gt; html = \u0026#39;\u0026lt;input type=\u0026#34;text\u0026#34; id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34;\u0026gt;\u0026#39; \u0026gt;\u0026gt;\u0026gt; pattern = r\u0026#39;\u0026lt;(.+?) type=\u0026#34;(.+?)\u0026#34; id=\u0026#34;(.+?)\u0026#34; name=\u0026#34;(.+?)\u0026#34;\u0026#39; # any character before a space for the tag name, likewise between quotes for the other groups \u0026gt;\u0026gt;\u0026gt; m = re.match(pattern, html) # storing the match result in a variable \u0026gt;\u0026gt;\u0026gt; m \u0026lt;re.Match object; span=(0, 41), match=\u0026#39;\u0026lt;input type=\u0026#34;text\u0026#34; id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34;\u0026#39;\u0026gt; The variable m stores the result, which is a Match object. By default, it shows the complete match, but we can explore more details of the object. The groups method shows all the extracted groups and group lets us return specific groups:\n\u0026gt;\u0026gt;\u0026gt; m.groups() # capture groups (\u0026#39;input\u0026#39;, \u0026#39;text\u0026#39;, \u0026#39;id_cpf\u0026#39;, \u0026#39;cpf\u0026#39;) \u0026gt;\u0026gt;\u0026gt; m.group(0) # entire match \u0026#39;\u0026lt;input type=\u0026#34;text\u0026#34; id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34;\u0026#39; \u0026gt;\u0026gt;\u0026gt; m.group(1) # first group \u0026#39;input\u0026#39; \u0026gt;\u0026gt;\u0026gt; m.group(2, 1, 3) # specific groups (\u0026#39;text\u0026#39;, \u0026#39;input\u0026#39;, \u0026#39;id_cpf\u0026#39;) Now suppose that the order of the HTML tag attributes can change:\n\u0026gt;\u0026gt;\u0026gt; html1 = \u0026#39;\u0026lt;input type=\u0026#34;text\u0026#34; id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34;\u0026gt;\u0026#39; \u0026gt;\u0026gt;\u0026gt; html2 = \u0026#39;\u0026lt;input id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34; type=\u0026#34;text\u0026#34;\u0026gt;\u0026#39; Clearly, our previous pattern will not work, since it depended heavily on the order of the attributes. We have already seen the | metacharacter, which indicates an alternative, something that will be useful here because we want to grab each attribute regardless of order.\nAnother concept we will use is the non-capturing group, represented by (?:...), replacing the ellipsis with a regular expression. To understand, consider the following examples:\n\u0026gt;\u0026gt;\u0026gt; m = re.match(\u0026#39;name=\u0026#34;(.+?)\u0026#34;\u0026#39;, \u0026#39;name=\u0026#34;Francisco\u0026#34;\u0026#39;) \u0026gt;\u0026gt;\u0026gt; m.groups() (\u0026#39;Francisco\u0026#39;,) \u0026gt;\u0026gt;\u0026gt; m = re.match(\u0026#39;name=\u0026#34;(?:.+?)\u0026#34;\u0026#39;, \u0026#39;name=\u0026#34;Francisco\u0026#34;\u0026#39;) \u0026gt;\u0026gt;\u0026gt; m.groups() () In the first case, we used the pattern in the group to get the value \u0026ldquo;Francisco\u0026rdquo; and in the second we excluded that group.\nWe can use this pattern alternately to get each group of the HTML tag regardless of order:\n\u0026gt;\u0026gt;\u0026gt; pattern = r\u0026#39;\u0026lt;(.+?) (?:(?:type=\u0026#34;(.+?)\u0026#34;|id=\u0026#34;(.+?)\u0026#34;|name=\u0026#34;(.+?)\u0026#34;) ?)*\u0026#39; # the outer group indicates the presence or not of a space \u0026gt;\u0026gt;\u0026gt; m = re.match(pattern, html1) \u0026gt;\u0026gt;\u0026gt; m \u0026lt;re.Match object; span=(0, 41), match=\u0026#39;\u0026lt;input type=\u0026#34;text\u0026#34; id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34;\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; m.groups() (\u0026#39;input\u0026#39;, \u0026#39;text\u0026#39;, \u0026#39;id_cpf\u0026#39;, \u0026#39;cpf\u0026#39;) \u0026gt;\u0026gt;\u0026gt; m = re.match(pattern, html2) \u0026gt;\u0026gt;\u0026gt; m \u0026lt;re.Match object; span=(0, 41), match=\u0026#39;\u0026lt;input id=\u0026#34;id_cpf\u0026#34; name=\u0026#34;cpf\u0026#34; type=\u0026#34;text\u0026#34;\u0026#39;\u0026gt; \u0026gt;\u0026gt;\u0026gt; m.groups() (\u0026#39;input\u0026#39;, \u0026#39;text\u0026#39;, \u0026#39;id_cpf\u0026#39;, \u0026#39;cpf\u0026#39;) Named groups # Finally, when there are several groups, it is useful to name them. Python has a specific way to indicate the name of each group: ?P\u0026lt;name\u0026gt;.\n\u0026gt;\u0026gt;\u0026gt; pattern = r\u0026#39;\u0026lt;(?P\u0026lt;tag\u0026gt;.+?) (?:(?:type=\u0026#34;(?P\u0026lt;type\u0026gt;.+?)\u0026#34;|id=\u0026#34;(?P\u0026lt;id\u0026gt;.+?)\u0026#34;|name=\u0026#34;(?P\u0026lt;name\u0026gt;.+?)\u0026#34;) ?)*\u0026#39; \u0026gt;\u0026gt;\u0026gt; m = re.match(pattern, html1) \u0026gt;\u0026gt;\u0026gt; m.groups() (\u0026#39;input\u0026#39;, \u0026#39;text\u0026#39;, \u0026#39;id_cpf\u0026#39;, \u0026#39;cpf\u0026#39;) When named groups are used, there is a very useful method, groupdict, which returns the group name and the value as a dictionary:\n\u0026gt;\u0026gt;\u0026gt; m.groupdict() {\u0026#39;tag\u0026#39;: \u0026#39;input\u0026#39;, \u0026#39;type\u0026#39;: \u0026#39;text\u0026#39;, \u0026#39;id\u0026#39;: \u0026#39;id_cpf\u0026#39;, \u0026#39;name\u0026#39;: \u0026#39;cpf\u0026#39;} \u0026gt;\u0026gt;\u0026gt; m = re.match(pattern, html2) \u0026gt;\u0026gt;\u0026gt; m.groups() (\u0026#39;input\u0026#39;, \u0026#39;text\u0026#39;, \u0026#39;id_cpf\u0026#39;, \u0026#39;cpf\u0026#39;) \u0026gt;\u0026gt;\u0026gt; m.groupdict() {\u0026#39;tag\u0026#39;: \u0026#39;input\u0026#39;, \u0026#39;type\u0026#39;: \u0026#39;text\u0026#39;, \u0026#39;id\u0026#39;: \u0026#39;id_cpf\u0026#39;, \u0026#39;name\u0026#39;: \u0026#39;cpf\u0026#39;} Conclusion and an IMPORTANT note # The important note is: just because something is possible to solve with regular expressions does not mean it should be solved with regular expressions.\nIn Python specifically, string manipulations involving substitutions are usually more effective with string methods such as replace. The documentation itself makes this clear.\nLikewise, even though I used the HTML tag case as an example, there are many online discussions about more efficient ways to extract information from HTML and why regex is generally not the best option. See here, here and here. Get ready for heated discussions. And use good judgment to know when using it is acceptable and when another tool should be used.\nRegular expressions are useful, but they can be very hard to read and debug. Try to keep them as short and as specific as possible. It is a much broader subject than what was presented here; I tried to include what I consider a good start and, obviously, what is within my knowledge. It is certainly a very valuable tool to know.\nSee you next time!\n","date":"2020-09-20","externalUrl":null,"permalink":"/2020/09/regex-regular-expressions-python/","section":"Blog","summary":"Knowing regular expressions, or simply regex, is very useful, but it tends to look a bit scary to a beginner. In this article we cover the basics of regex using Python","title":"RegEx - Regular Expressions in Python","type":"posts"},{"content":"You may have been in this situation: you are writing a paper or doing research and find a figure that presents interesting data, but it is low quality or you would like to add or remove elements from it. However, the data that originated it are not available. How do you get out of this? That is what we will see in today\u0026rsquo;s article.\nIf you prefer video, click the player below (in Portuguese). The full article follows the video.\nOriginal figure # When I was preparing the article about the case of the airplane that ran out of fuel in mid-flight, I found a US Department of Defense document about aircraft fuels. I was looking for it because, as I described in the article, fuel density varies with temperature and the whole problem was caused by an error in the unit used for the density value. Originally, I planned to include a chart showing how density varies with temperature, but later I thought it would not add much to the story and left it out. But now I will use the chart here to show how we can obtain data from figures and considerably improve their presentation.\nLet us start by looking at the original figure from the Department of Defense document:\nAdmittedly, the quality is very poor. The original material, which was already old (the document is from 1983), was scanned not very carefully, making the chart\u0026rsquo;s grid lines hard to see. Obviously, without such lines, the usability of the chart is reduced. The document does not provide a table of values. Had such a table been provided, rebuilding the chart would be easier: any spreadsheet could be used for that. Since no such table is available, we need to extract the data directly from the chart.\nObviously, one way to do this would be manually. That is, reading a few points of each line of the chart and rebuilding the lines from those points. The first problem with this approach is that it is slow. The second is that it is more error-prone, especially because of how hard it is to read some regions of the figure. Finally, this approach is even reasonable for this chart, since there is clearly a linear profile, making it easy to extrapolate the general behavior of each data set from a few points. However, if the behavior followed another profile, more points would be needed for a more accurate reconstruction. Thus, a software-based approach proves more viable and more cost-effective.\nThe Engauge Digitizer program # Engauge Digitizer is a free, open-source program designed to recover data from image files. The data can then be saved and used as input to programs that build charts. The following figure, taken from the project\u0026rsquo;s website, shows exactly the process:\nEngauge Digitizer process The program is cross-platform, available in the app stores of most Linux distributions and on the Mac. Installers for those systems and also for Windows are available in the project repository.\nUsing the program is quite simple: you just import the figure and follow a few steps. It is easier to explain with a video. At the top of this article there is the full video (in Portuguese).\nRedoing the chart with Python # Now that we have a file with the data, we can import it into any spreadsheet program or use a programming language. Since I like having more creative freedom, I will use Python to rebuild the chart. Below is a Jupyter Notebook with the steps I followed. If you want to download the Notebook, click here to go to Ciência Programada\u0026rsquo;s GitHub and download the file.\nFirst, let us import the libraries we will use:\nimport numpy as np from scipy.stats import linregress import matplotlib.pyplot as plt The role of each library will be shown throughout the file.\nLet us save the path to the data file in a variable. This makes it easier to change the code if the file is later moved to another folder:\ndata_file = \u0026#39;dados_combustivel.csv\u0026#39; numpy has the genfromtxt method, which extracts data from text files. The delimiter parameter specifies the delimiter used in the file, which here is a comma since it is a csv format; the names parameter lets us set the names of the data columns. The value True means to use the line right above the numeric data in the source file:\ndata = np.genfromtxt(data_file, delimiter=\u0026#39;,\u0026#39;, names=True) We can view the available columns. Each data set can be obtained from the variable by column name with the syntax data[column_name]:\ndata.dtype.names ('x', 'JP5', 'JetA', 'JP7', 'JetB', 'AvGas') Engauge Digitizer exported points from our source chart. We can then plot a first draft of the chart with each set of points using Matplotlib\u0026rsquo;s scatter method to get an idea of the general behavior of the data:\nfor name in data.dtype.names[1:]: plt.scatter(data[\u0026#39;x\u0026#39;], data[name], label=name) plt.legend() plt.show() It is quite noticeable that each data set has linear behavior. This is where the linregress method imported from the SciPy library comes in.\nLet us take a look, for example, at the linear regression data for the data set of the JetA fuel:\nlinregress(data[\u0026#39;x\u0026#39;], data[\u0026#39;JetA\u0026#39;]) LinregressResult(slope=-0.7221033469533403, intercept=819.8747195466843, rvalue=-0.9999442604546929, pvalue=0.0, stderr=0.0005972008546052884) Note that the method returns five values that describe the regression: the slope of the line, the intercept, the R value (correlation coefficient), the P value, and the standard error. A detailed description of the meaning of each parameter can be found in the documentation. Between us, what matters is checking that the R value, or more specifically the square of R, is as close to 1 as possible and that the standard error is low, so that we can really consider the data to behave linearly.\nLet us check each data set with a loop:\nfor name in data.dtype.names[1:]: regression = linregress(data[\u0026#39;x\u0026#39;], data[name]) print(f\u0026#34;Regression for {name}: R^2={regression.rvalue**2:.4f} and stderr={regression.stderr:.4f}\u0026#34;) Regression for JP5: R^2=0.9998 and stderr=0.0007 Regression for JetA: R^2=0.9999 and stderr=0.0006 Regression for JP7: R^2=0.9998 and stderr=0.0008 Regression for JetB: R^2=0.9981 and stderr=0.0026 Regression for AvGas: R^2=0.9999 and stderr=0.0007 Therefore, we can see that treating each set as linear is quite reasonable.\nWe can then plot the line resulting from the regression of each set and generate our final chart. Let us use one of Matplotlib\u0026rsquo;s predefined chart styles, which can be seen here. And let us also add titles to the axes and to the chart:\n# range of temperature values for the chart temperature = np.arange(-40, 100, 10) plt.style.use(\u0026#39;ggplot\u0026#39;) for name in data.dtype.names[1:]: regression = linregress(data[\u0026#39;x\u0026#39;], data[name]) plt.plot(temperature, regression.slope * temperature + regression.intercept, label=name) plt.legend() plt.xlabel(\u0026#39;Temperature / °C\u0026#39;) plt.ylabel(\u0026#39;Density / kg/m³\u0026#39;) plt.title(\u0026#39;Aircraft fuel density vs temperature\u0026#39;) plt.show() It is a much more presentable chart than the original, do you agree? And many other customizations can be made at the user\u0026rsquo;s discretion. Below is a more elaborate chart, using a few more Matplotlib features:\n# changing the tick frequency on the axes import matplotlib.ticker as plticker density_ticks = plticker.MultipleLocator(base=20) temperature_ticks = plticker.MultipleLocator(base=10) plt.style.use(\u0026#39;ggplot\u0026#39;) fig, ax = plt.subplots(figsize=(6, 8), constrained_layout=True) for name in data.dtype.names[1:]: regression = linregress(data[\u0026#39;x\u0026#39;], data[name]) ax.plot(temperature, regression.slope * temperature + regression.intercept, label=name, linewidth=5) ax.legend(fontsize=14) ax.xaxis.set_major_locator(temperature_ticks) ax.yaxis.set_major_locator(density_ticks) ax.tick_params(labelsize=13) ax.minorticks_on() ax.grid(which=\u0026#39;minor\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=\u0026#39;1.0\u0026#39;, color=\u0026#39;white\u0026#39;) ax.set_xlabel(\u0026#39;Temperature / °C\u0026#39;, fontsize=15) ax.set_ylabel(\u0026#39;Density / kg/m³\u0026#39;, fontsize=15) ax.set_title(\u0026#39;Aircraft fuel density vs temperature\u0026#39;, fontsize=17) plt.show() Conclusion # Engauge Digitizer is a very useful tool, especially for those who frequently deal with charts and do not always have the original data available. It helps everyone from those who simply want a more presentable chart for a school assignment to professors and researchers, or even organizations that deal with large image databases. Its power, combined with a robust graphical tool such as Matplotlib, makes it possible to turn poor-quality charts into high-resolution, much more informative images.\nSee you next time.\n","date":"2020-09-15","externalUrl":null,"permalink":"/2020/09/digitizing-charts-engauge-digitizer-python/","section":"Blog","summary":"You may have been in this situation: you are writing a paper or doing research and find a figure that presents interesting data, but it is low quality or you would like to add or remove elements from it. However, the data that originated it are not available. How do you get out of this? That is what we will see in today’s article.","title":"Digitizing Charts with Engauge Digitizer and Python","type":"posts"},{"content":"","date":"2020-09-15","externalUrl":null,"permalink":"/categories/reviews/","section":"Categories","summary":"","title":"Reviews","type":"categories"},{"content":" The Jamaican Usain Bolt is considered by many to be the greatest sprinter of all time. Even though he retired from the track in 2017, his name is still one of the first to come to the general public\u0026rsquo;s mind when the subject is track and field.\nIn this article, we will see how to use Python to find the speed and acceleration the Jamaican reached in the 100 m races of the 2008 Beijing Olympics, where he set the world record, and of the 2009 World Championships in Berlin, where he broke his own record. We will also take the chance to discuss mathematical models and the care needed in choosing them and interpreting their results.\nThe code for the article is in Jupyter Notebook format in the Ciência Programada repository on GitHub. Take a look and practice by downloading the file and changing the code.\nIf you prefer to watch a video (in Portuguese), click on the player below. The full article follows the video.\nGetting the data # The first step is, obviously, to get the data and put it in a form that the packages we will use can handle. The total time of each race is relatively easy to find on the internet but, to get a better feel for the athlete\u0026rsquo;s performance in the race, the more detail we get, the better.\nThe site Speed Endurance compiled the data from the reports of the International Association of Athletics Federations (IAAF) for both events. Here is an adapted table with the compiled data, with times in seconds for Beijing 2008 and Berlin 2009:\nm Beijing split Beijing total Berlin split Berlin total Diff. TR 0.165 0.146 −0.019 0‑10 1.85 1.85 1.89 1.89 0.04 10‑20 1.02 2.87 0.99 2.88 0.01 20‑30 0.91 3.78 0.90 3.78 0.00 30‑40 0.87 4.65 0.86 4.64 −0.01 40‑50 0.85 5.50 0.83 5.47 −0.03 50‑60 0.82 6.32 0.82 6.29 −0.03 60‑70 0.82 7.14 0.81 7.10 −0.04 70‑80 0.82 7.96 0.82 7.92 −0.04 80‑90 0.83 8.79 0.83 8.75 −0.04 90‑100 0.90 9.69 0.83 9.58 −0.11 In the table, TR stands for \u0026ldquo;reaction time\u0026rdquo; (the Portuguese \u0026ldquo;tempo de reação\u0026rdquo;), that is, the time that passed from the starting gun until the athlete actually moved. There was no wind in the 2008 race and there was a tailwind of 0.9 m/s in the 2009 race, according to the data in the reports.\nThe interesting thing is that we have Bolt\u0026rsquo;s times every 10 m, which will be useful later. We can take the cumulative times, put them in a spreadsheet and export them to the csv format (comma separated values), a format usually recognized by any mathematical package.\nLet\u0026rsquo;s take a look at our file:\n!cat bolt.csv Splits,\u0026quot;2008 Beijing\u0026quot;,\u0026quot;2009 Berlin\u0026quot; 0,0,0 \u0026quot;0 - 10\u0026quot;,1.85,1.89 \u0026quot;10 - 20\u0026quot;,2.87,2.88 \u0026quot;20 - 30\u0026quot;,3.78,3.78 \u0026quot;30 - 40\u0026quot;,4.65,4.64 \u0026quot;40 - 50\u0026quot;,5.50,5.47 \u0026quot;50 - 60\u0026quot;,6.32,6.29 \u0026quot;60 - 70\u0026quot;,7.14,7.10 \u0026quot;70 - 80\u0026quot;,7.96,7.92 \u0026quot;80 - 90\u0026quot;,8.79,8.75 \u0026quot;90 - 100\u0026quot;,9.69,9.58 Split is the term used for each division of the race, each 10-meter mark. Notice that I put a row with time 0 at the beginning to make it easier to build plots later. As with everything in programming, the default language is English, so the numbers use the English format, with a point separating the decimal part. I strongly recommend setting your programs to use the point as the default, since not every package will accept files with a comma.\nLet\u0026rsquo;s store the file path in a variable to make it easier to refer to during the program:\ndata_file = \u0026#39;bolt.csv\u0026#39; One of the most widely used packages for dealing with numerical data and scientific computing in Python is NumPy. Let\u0026rsquo;s import it together with Matplotlib, for plots:\nimport numpy as np import matplotlib.pyplot as plt To import data from the csv file, we will use the genfromtxt method. To understand how this method works, let\u0026rsquo;s look at the output when passing only the file and the delimiter (a comma in the case of csv files):\nnp.genfromtxt(data_file, delimiter=\u0026#39;,\u0026#39;) array([[ nan, nan, nan], [0. , 0. , 0. ], [ nan, 1.85, 1.89], [ nan, 2.87, 2.88], [ nan, 3.78, 3.78], [ nan, 4.65, 4.64], [ nan, 5.5 , 5.47], [ nan, 6.32, 6.29], [ nan, 7.14, 7.1 ], [ nan, 7.96, 7.92], [ nan, 8.79, 8.75], [ nan, 9.69, 9.58]]) Notice that there is nan in several positions. More specifically, in the positions where there was text. That is because, by default, the method expects type float in every position. When it finds something unexpected, of another type, it replaces it with nan (not a number). So we need to tell it that the first column is text (type string) and, in addition, that the first row contains the names of each column. For that, we use the following parameters of the method:\nnp.genfromtxt(data_file, delimiter=\u0026#39;,\u0026#39;, dtype=(str, float, float), names=True) array([('', 0. , 0. ), ('', 1.85, 1.89), ('', 2.87, 2.88), ('', 3.78, 3.78), ('', 4.65, 4.64), ('', 5.5 , 5.47), ('', 6.32, 6.29), ('', 7.14, 7.1 ), ('', 7.96, 7.92), ('', 8.79, 8.75), ('', 9.69, 9.58)], dtype=[('Splits', '\u0026lt;U'), ('2008_Beijing', '\u0026lt;f8'), ('2009_Berlin', '\u0026lt;f8')]) Notice that the names part worked, now the output lists the names at the end. However, the texts appear as empty strings. According to the documentation we must pass the encoding type and the expected number of characters. In our case, unicode and 10 characters:\nnp.genfromtxt(data_file, delimiter=\u0026#39;,\u0026#39;, dtype=(\u0026#39;U10\u0026#39;, float, float), names=True) array([('0', 0. , 0. ), ('\u0026quot;0 - 10\u0026quot;', 1.85, 1.89), ('\u0026quot;10 - 20\u0026quot;', 2.87, 2.88), ('\u0026quot;20 - 30\u0026quot;', 3.78, 3.78), ('\u0026quot;30 - 40\u0026quot;', 4.65, 4.64), ('\u0026quot;40 - 50\u0026quot;', 5.5 , 5.47), ('\u0026quot;50 - 60\u0026quot;', 6.32, 6.29), ('\u0026quot;60 - 70\u0026quot;', 7.14, 7.1 ), ('\u0026quot;70 - 80\u0026quot;', 7.96, 7.92), ('\u0026quot;80 - 90\u0026quot;', 8.79, 8.75), ('\u0026quot;90 - 100\u0026quot;', 9.69, 9.58)], dtype=[('Splits', '\u0026lt;U10'), ('2008_Beijing', '\u0026lt;f8'), ('2009_Berlin', '\u0026lt;f8')]) Now it works, but notice that we did not store the data in any variable. We can use a little Notebook trick. When we \u0026ldquo;forget\u0026rdquo; to store the result of the last executed cell in a variable, we can create the variable and assign _ to it:\ndata = _ data array([('0', 0. , 0. ), ('\u0026quot;0 - 10\u0026quot;', 1.85, 1.89), ('\u0026quot;10 - 20\u0026quot;', 2.87, 2.88), ('\u0026quot;20 - 30\u0026quot;', 3.78, 3.78), ('\u0026quot;30 - 40\u0026quot;', 4.65, 4.64), ('\u0026quot;40 - 50\u0026quot;', 5.5 , 5.47), ('\u0026quot;50 - 60\u0026quot;', 6.32, 6.29), ('\u0026quot;60 - 70\u0026quot;', 7.14, 7.1 ), ('\u0026quot;70 - 80\u0026quot;', 7.96, 7.92), ('\u0026quot;80 - 90\u0026quot;', 8.79, 8.75), ('\u0026quot;90 - 100\u0026quot;', 9.69, 9.58)], dtype=[('Splits', '\u0026lt;U10'), ('2008_Beijing', '\u0026lt;f8'), ('2009_Berlin', '\u0026lt;f8')]) We can access each column of the data by its name:\ndata[\u0026#39;Splits\u0026#39;] array(['0', '\u0026quot;0 - 10\u0026quot;', '\u0026quot;10 - 20\u0026quot;', '\u0026quot;20 - 30\u0026quot;', '\u0026quot;30 - 40\u0026quot;', '\u0026quot;40 - 50\u0026quot;', '\u0026quot;50 - 60\u0026quot;', '\u0026quot;60 - 70\u0026quot;', '\u0026quot;70 - 80\u0026quot;', '\u0026quot;80 - 90\u0026quot;', '\u0026quot;90 - 100\u0026quot;'], dtype='\u0026lt;U10') data[\u0026#39;2008_Beijing\u0026#39;] array([0. , 1.85, 2.87, 3.78, 4.65, 5.5 , 6.32, 7.14, 7.96, 8.79, 9.69]) data[\u0026#39;2009_Berlin\u0026#39;] array([0. , 1.89, 2.88, 3.78, 4.64, 5.47, 6.29, 7.1 , 7.92, 8.75, 9.58]) Now we have what we need to start exploring this data. Let\u0026rsquo;s begin with some simple plots.\nFirst plots # Let\u0026rsquo;s start with a simple plot of position versus time. For the data from the Berlin World Championships, we have:\nplt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], data[\u0026#39;Splits\u0026#39;]) Although in the table the information in interval form is easier for the reader to interpret, in my opinion in the plot it is better to use the whole value at the end of the interval. That is, 0, 10, 20\u0026hellip; So, let\u0026rsquo;s create a variable marks that will store these marks every 10 meters so we can use it as the vertical axis of the plot. For that, we will use the arange method which, according to the documentation, generates evenly spaced values in the form [start, stop). Notice, closed at the start and open at the end. That is why the code below has 101, so that the value 100 is included:\nmarks = np.arange(0, 101, 10) marks array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100]) Plotting again:\nplt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks) plt.show() It is much easier to understand, in my opinion. But numbers alone mean nothing. It would be better to put a title on the plot and say what each axis is about. In addition, grid lines would make it easier for the reader to interpret the plot. Let\u0026rsquo;s solve each case.\nMatplotlib has some predefined plot styles that can be seen here. Let\u0026rsquo;s choose one that shows grid lines:\nplt.style.use(\u0026#39;ggplot\u0026#39;) This style will apply to the whole Notebook from the moment the cell is executed.\nNow, a bit of basic Matplotlib to label the plot and the axes:\nplt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks) plt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Position / m\u0026#39;) plt.title(\u0026#39;Usain Bolt - Berlin World Championships - 2009\u0026#39;) plt.show() We can compare the data from the two competitions on the same plot:\nplt.scatter(data[\u0026#39;2008_Beijing\u0026#39;], marks, label=\u0026#39;2008 - Beijing\u0026#39;) plt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;) plt.legend() plt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Position / m\u0026#39;) plt.title(\u0026#39;Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() Notice that in the plot it is easier to see that Bolt started worse in the 2009 race (the stretch between 0 and 3 seconds), but recovered in the second half to break his own record.\nWe can also show the plots side by side. If you have trouble understanding the following code, read this article where I also present a bit of Matplotlib and explain a little how to work with axes.\n# useful to change the frequency of the ticks on the axes import matplotlib.ticker as plticker # creating the grid fig1, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) # assigning axes to each position of the grid ax1 = axarr[0] ax2 = axarr[1] # ticks on the position axis every 10 and on the time axis every 1 position_ticks = plticker.MultipleLocator(base=10) time_ticks = plticker.MultipleLocator(base=1) # the plots themselves ax1.scatter(data[\u0026#39;2008_Beijing\u0026#39;], marks, label=\u0026#39;2008 - Beijing\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;, color=\u0026#39;blue\u0026#39;) # naming the axes and placing the ticks at the desired interval for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Position / m\u0026#39;) ax.yaxis.set_major_locator(position_ticks) ax.legend() # main title fig1.suptitle(\u0026#39;Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() Since the position axis is the same in both plots, they could be merged:\nfig2, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), sharey=True, facecolor=(1,1,1)) # removing horizontal space between the plots fig2.subplots_adjust(wspace=0) ax1 = axarr[0] ax2 = axarr[1] position_ticks = plticker.MultipleLocator(base=10) time_ticks = plticker.MultipleLocator(base=1) ax1.scatter(data[\u0026#39;2008_Beijing\u0026#39;], marks, label=\u0026#39;2008 - Beijing\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;, color=\u0026#39;blue\u0026#39;) ax1.set_ylabel(\u0026#39;Position / m\u0026#39;) ax1.yaxis.set_major_locator(position_ticks) for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.legend() fig2.suptitle(\u0026#39;Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() I don\u0026rsquo;t like this form very much, but I\u0026rsquo;m leaving it here for your knowledge. If you are going to put plots on web pages, remember that many people view them on a phone, so it is better to keep the axes separate to help those who zoom in to see the plots separately. Merging axes is usually a better option in work meant to be read on a computer or on paper.\nHow do we \u0026ldquo;connect the dots\u0026rdquo;? # Notice that, so far, we have only plotted the points for each of Bolt\u0026rsquo;s positions in time in each race. But how do we connect these points? Or, those more familiar with Matplotlib must be wondering why I did not use the plot method:\n# WRONG!! plt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Position / m\u0026#39;) plt.title(\u0026#39;Usain Bolt - Berlin World Championships - 2009\u0026#39;) plt.plot(data[\u0026#39;2009_Berlin\u0026#39;], marks, marker=\u0026#39;o\u0026#39;) plt.show() Well, it is easy to see the problem with the plot above. Did you notice what the method did? It simply connected the dots, much like what you did as a little kid in those drawing books with dots to connect to form pictures. But later, when you became a slightly smarter kid, you began to notice that, for the picture to look aesthetically better, it was not always right to connect with straight lines, was it?\nSo, this is not a connect-the-dots drawing, but the idea is the same. Should we connect with straight lines? And here the criterion is obviously not aesthetic but a logical and physical analysis of the situation.\nLogically, Bolt did not run at constant speed throughout the race. Especially in the initial start, where he left from rest (zero speed), he certainly accelerated using all possible muscular explosion until reaching a maximum speed and tried to hold that maximum until the end of the race, but may not have managed to. By the way, let\u0026rsquo;s watch the races.\nIn the video of the 2008 race, it is noticeable that he was already celebrating in the final meters:\nIn 2009 he did not celebrate early, he just looked at the clock in the final strides:\nIt is to be expected that the celebration in 2008 slowed him down a bit.\nAnd what does speed have to do with our position plot? Well, speed is the change in position over time. So, more position points would be useful for a better study of the speed. This is where the concept of interpolation comes in.\nInterpolation # We need to estimate points between the experimental points we have. The formal name for this is interpolation. Do not confuse it with regression. In a regression, we try to determine which mathematical function best describes a given set of data, not necessarily passing through points of that data. In interpolation you pass through the points.\nFor the purposes of this article, this definition of interpolation is enough. If you want more rigor, see this article on interpolation, this one on regression and this discussion about the difference between the two concepts.\nTo do the interpolation we will use the SciPy library, which has numerical routines for statistics, linear algebra, optimization problems, regression and interpolation.\nThe subject is so broad that a look at the documentation shows the existence of several interpolation methods. But let\u0026rsquo;s go step by step.\nFirst we will show that the Matplotlib plot method really does perform a linear interpolation between each pair of points. We will use the interp1d package from SciPy.\nfrom scipy.interpolate import interp1d Let\u0026rsquo;s then linearly interpolate our points. To keep the article shorter, I will do it only for the 2009 data:\nlinear_2009_berlin = interp1d(data[\u0026#39;2009_Berlin\u0026#39;], marks, kind=\u0026#39;linear\u0026#39;, fill_value=\u0026#39;extrapolate\u0026#39;) The kind='linear' is self-explanatory. The fill_value parameter decides what to do if values outside the interpolation range are passed to the resulting function. In this case, I asked it to extrapolate.\nNow that we have our interpolating function, we need to pass x values to it, in our case time. Let\u0026rsquo;s create an array of times spaced 0.1 s apart:\ntime = np.arange(0, 9.7, 0.1) Plotting:\nplt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Position / m\u0026#39;) plt.title(\u0026#39;Usain Bolt - Berlin World Championships - 2009\u0026#39;) plt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;) plt.plot(time, linear_2009_berlin(time), label=\u0026#39;Linear\u0026#39;) plt.legend() plt.show() It is the same plot generated by the plot method.\nAlright, we showed that a linear interpolation was indeed made between each pair of points. But what should we do then?\nHere it is worth noting that there are several approaches to interpolation. One approach is to look for a polynomial that passes through all the points. Given n points, there is a polynomial of degree n-1 that interpolates them. This approach is computationally expensive and can exhibit some oscillatory artifacts at the endpoints, which is known as Runge\u0026rsquo;s phenomenon.\nThis interactive animation illustrates this phenomenon well for those who had trouble visualizing what is written above.\nAnother approach is piecewise interpolation (also called interpolation by sectioned polynomials), that is, instead of looking for a single function that passes through all the points, the points are divided into intervals. Each interval is interpolated and attached to the next one so as to obtain a continuous curve.\nSplines # Among the forms of piecewise interpolation, the so-called spline stands out. The advantage is that good interpolations are obtained with low-degree polynomials, avoiding Runge\u0026rsquo;s phenomenon.\nA good starting point is cubic splines, that is, using degree 3 polynomials on each interval. It is a low degree and it guarantees continuity of the function up to its second derivative, something that will be important for us later. It is worth noting that it is a common starting point when you know little about the behavior of the data, but it is not a silver bullet, so always try to know the context of your data as much as possible.\nEven among cubic splines there are different types, depending on the boundary conditions, but I will not go into that in this article. We will use a specific SciPy method for cubic splines, CubicSpline. As you can see in the linked documentation, it is possible to change the type (boundary conditions), but I will keep the default here in this article.\nfrom scipy.interpolate import CubicSpline Let\u0026rsquo;s then get the splines for our data:\ncs_2009_berlin = CubicSpline(data[\u0026#39;2009_Berlin\u0026#39;], marks) cs_2008_beijing = CubicSpline(data[\u0026#39;2008_Beijing\u0026#39;], marks) fig3, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] position_ticks = plticker.MultipleLocator(base=10) time_ticks = plticker.MultipleLocator(base=1) ax1.scatter(data[\u0026#39;2008_Beijing\u0026#39;], marks, label=\u0026#39;2008 - Beijing\u0026#39;, color=\u0026#39;red\u0026#39;) ax1.plot(time, cs_2008_beijing(time), label=\u0026#39;Cubic Spline\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;, color=\u0026#39;blue\u0026#39;) ax2.plot(time, cs_2009_berlin(time), label=\u0026#39;Cubic Spline\u0026#39;, color=\u0026#39;blue\u0026#39;) # naming the axes and placing the ticks at the desired interval for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Position / m\u0026#39;) ax.yaxis.set_major_locator(position_ticks) ax.legend() # main title fig3.suptitle(\u0026#39;Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() Finally, it looks like we now have a plot that lets us continue our explorations.\nGetting Bolt\u0026rsquo;s speed and acceleration # Average speed # Now that we have what looks like a more acceptable model of how Bolt\u0026rsquo;s position varies in time, we can calculate his speed. But we need to clarify what kind of speed we want to determine. After all, if it is only the average speed, that could have been calculated already, since we know the total displacement, 100 meters, and the time the displacement lasted, the final time of the race, which is the last position of each time array:\nmean_speed_beijing = 100 / data[\u0026#39;2008_Beijing\u0026#39;][-1] print(f\u0026#39;Mean speed in Beijing (2008): {mean_speed_beijing:.2f} m/s\u0026#39;) mean_speed_berlin = 100 / data[\u0026#39;2009_Berlin\u0026#39;][-1] print(f\u0026#39;Mean speed in Berlin (2009): {mean_speed_berlin:.2f} m/s\u0026#39;) Mean speed in Beijing (2008): 10.32 m/s Mean speed in Berlin (2009): 10.44 m/s I have already written here about the pint package which lets you deal with units and make conversions. Let\u0026rsquo;s use this package to convert the values to kilometers per hour, the unit we are most used to:\nimport pint ureg = pint.UnitRegistry(fmt_locale=\u0026#39;en_US\u0026#39;) Q_ = ureg.Quantity mean_speed_beijing = Q_(mean_speed_beijing, \u0026#39;m/s\u0026#39;) mean_speed_berlin = Q_(mean_speed_berlin, \u0026#39;m/s\u0026#39;) print(f\u0026#34;Mean speed in Beijing (2008): {mean_speed_beijing.to(\u0026#39;km/hour\u0026#39;):.2f}\u0026#34;) print(f\u0026#34;Mean speed in Berlin (2009): {mean_speed_berlin.to(\u0026#39;km/hour\u0026#39;):.2f}\u0026#34;) Mean speed in Beijing (2008): 37.15 kilometers per hour Mean speed in Berlin (2009): 37.58 kilometers per hour Impressive, right?\nInstantaneous speed # But what would be more interesting is to estimate the speed at each moment of the race, the so-called instantaneous speed. After all, as we already discussed, Bolt\u0026rsquo;s speed during the races is not constant. To get such averages, at some moments Bolt certainly ran above those values. How can we get instantaneous speeds?\nWe will use a bit of calculus here and the fact that we now have a curve joining the experimental points. For the average speed, we took two distant points, the start and the end of the race. Now, imagine taking points closer and closer together, that is, making the time interval between the points smaller and smaller. In the limit where this interval tends to zero, we get the instantaneous speed.\nIt is easier to visualize with an image. Watch the animation below. In it we have a curve of some function, two marked points, A and B, and the triangle formed to obtain the change in the y-axis quantity and the x-axis one, with the help of a third point C. In the case of our plots, change in distance and in time, respectively. In the animation, the points get closer and closer. Notice that, in the limit where the points become closer, the hypotenuse of the triangle, initially a secant line of the curve, becomes a tangent line at the point of virtual meeting A. The blue line is the tangent at point A, shown precisely to show how the secant gets closer and closer to a tangent.\nAnimation showing a secant line approaching the tangent line at a point of a curve So, mathematically, the instantaneous speed at a given moment is the slope of the line tangent to the curve of the position vs time plot at the desired moment. In calculus, this is the same as saying that the instantaneous speed is the derivative of position with respect to time at a given point of the plot.\nI mentioned earlier that the fact that cubic splines guarantee that the resulting curve has first and second derivatives was important. Now we know why.\nThe nice thing about SciPy\u0026rsquo;s CubicSpline is that it is very simple to ask for the first derivative, just by passing the parameter 1 to the spline we obtained. See the code below:\nfig4, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] speed_ticks = plticker.MultipleLocator(base=2) time_ticks = plticker.MultipleLocator(base=1) ax1.plot(time, cs_2008_beijing(time, 1), label=\u0026#39;Cubic Spline - 1st derivative\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.plot(time, cs_2009_berlin(time, 1), label=\u0026#39;Cubic Spline - 1st derivative\u0026#39;, color=\u0026#39;blue\u0026#39;) for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Speed / m/s\u0026#39;) ax.yaxis.set_major_locator(speed_ticks) ax.legend() fig4.suptitle(\u0026#39;Speed - Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() Notice that we got profiles consistent with what we expected. In the 2008 race, the speed drops near the final second of the race, when Bolt was visibly celebrating already. In 2009, the Jamaican is more consistent in speed.\nWe can check the maximum speeds our model gives for each case:\nmax_speed_2008_cs = Q_(max(cs_2008_beijing(time, 1)), \u0026#39;m/s\u0026#39;) max_speed_2009_cs = Q_(max(cs_2009_berlin(time, 1)), \u0026#39;m/s\u0026#39;) print(f\u0026#39;{max_speed_2008_cs:.2f}\u0026#39;) print(f\u0026#39;{max_speed_2009_cs:.2f}\u0026#39;) 12.26 meters per second 12.37 meters per second In kilometers per hour:\nprint(f\u0026#34;{max_speed_2008_cs.to(\u0026#39;km/hour\u0026#39;):.2f}\u0026#34;) print(f\u0026#34;{max_speed_2009_cs.to(\u0026#39;km/hour\u0026#39;):.2f}\u0026#34;) 44.14 kilometers per hour 44.53 kilometers per hour That is enough to get him a ticket in urban areas. Did anyone write down the license plate?\nFollowing the logic of everything we have shown so far, we can get a curve for acceleration. Acceleration is the change in speed over time, which is the same as saying that acceleration is the first derivative of speed with respect to time. Or, the second derivative of position with respect to time. Hence:\nfig5, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] speed_ticks = plticker.MultipleLocator(base=1) time_ticks = plticker.MultipleLocator(base=1) ax1.plot(time, cs_2008_beijing(time, 2), label=\u0026#39;Cubic Spline - 2nd derivative\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.plot(time, cs_2009_berlin(time, 2), label=\u0026#39;Cubic Spline - 2nd derivative\u0026#39;, color=\u0026#39;blue\u0026#39;) for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Acceleration / m/s²\u0026#39;) ax.yaxis.set_major_locator(speed_ticks) ax.legend() fig5.suptitle(\u0026#39;Acceleration - Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() These plots are also relatively coherent: a large initial acceleration and then a positive acceleration that keeps trying to increase the speed. In 2008, a negative acceleration very close to the end, indicating deceleration near the finish line.\nAlthough they are coherent, we need to be careful with our models. In the acceleration plots we observe abrupt changes in the profiles. This is a bit strange, remember that this is a human, there is no accelerator or brake pedal! Perhaps we should expect less abrupt variations.\nImproving our model # Obviously, many people and companies study athletes\u0026rsquo; performance. There is even a whole field of study called sports biomechanics devoted to it. Looking at studies in the area, we see that splines are usually used, but of degree 5, not 3. The term is quintic splines, with quintic referring to degree 5. It is no coincidence that there is a company called Quintic that works precisely with sports biomechanics software. Some biomechanics books, like this one, have sections devoted to the study of degree 5 splines given their importance in the area.\nSo, we can check what an interpolation with fifth-degree splines would look like. SciPy has UnivariateSpline, where we can pass a parameter to indicate the desired degree of the spline:\nfrom scipy.interpolate import UnivariateSpline s_degree5_2008_beijing = UnivariateSpline(data[\u0026#39;2008_Beijing\u0026#39;], marks, k=5) s_degree5_2009_berlin = UnivariateSpline(data[\u0026#39;2009_Berlin\u0026#39;], marks, k=5) Let\u0026rsquo;s make the plots of position, speed and acceleration versus time:\nfig6, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] position_ticks = plticker.MultipleLocator(base=10) time_ticks = plticker.MultipleLocator(base=1) ax1.scatter(data[\u0026#39;2008_Beijing\u0026#39;], marks, label=\u0026#39;2008 - Beijing\u0026#39;, color=\u0026#39;red\u0026#39;) ax1.plot(time, s_degree5_2008_beijing(time), label=\u0026#39;Degree 5 spline\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.scatter(data[\u0026#39;2009_Berlin\u0026#39;], marks, label=\u0026#39;2009 - Berlin\u0026#39;, color=\u0026#39;blue\u0026#39;) ax2.plot(time, s_degree5_2009_berlin(time), label=\u0026#39;Degree 5 spline\u0026#39;, color=\u0026#39;blue\u0026#39;) # naming the axes and placing the ticks at the desired interval for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Position / m\u0026#39;) ax.yaxis.set_major_locator(position_ticks) ax.legend() # main title fig6.suptitle(\u0026#39;Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() fig7, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] speed_ticks = plticker.MultipleLocator(base=2) time_ticks = plticker.MultipleLocator(base=1) ax1.plot(time, s_degree5_2008_beijing(time, 1), label=\u0026#39;Degree 5 spline - 1st derivative\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.plot(time, s_degree5_2009_berlin(time, 1), label=\u0026#39;Degree 5 spline - 1st derivative\u0026#39;, color=\u0026#39;blue\u0026#39;) for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Speed / m/s\u0026#39;) ax.yaxis.set_major_locator(speed_ticks) ax.legend() fig7.suptitle(\u0026#39;Speed - Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() fig8, axarr = plt.subplots(nrows=1, ncols=2, figsize=(10,5), constrained_layout=True, facecolor=(1,1,1)) ax1 = axarr[0] ax2 = axarr[1] speed_ticks = plticker.MultipleLocator(base=1) time_ticks = plticker.MultipleLocator(base=1) ax1.plot(time, s_degree5_2008_beijing(time, 2), label=\u0026#39;Degree 5 spline - 2nd derivative\u0026#39;, color=\u0026#39;red\u0026#39;) ax2.plot(time, s_degree5_2009_berlin(time, 2), label=\u0026#39;Degree 5 spline - 2nd derivative\u0026#39;, color=\u0026#39;blue\u0026#39;) for ax in axarr: ax.set_xlabel(\u0026#39;Time / s\u0026#39;) ax.xaxis.set_major_locator(time_ticks) ax.set_ylabel(\u0026#39;Acceleration / m/s²\u0026#39;) ax.yaxis.set_major_locator(speed_ticks) ax.legend() fig8.suptitle(\u0026#39;Acceleration - Usain Bolt - Beijing (2008) vs Berlin (2009)\u0026#39;) plt.show() Notice that the acceleration curves are now smoother. More consistent with a gradual variation by the athlete. In fact, contrary to the previous cubic spline, the 2009 plot now shows a slight deceleration at the end.\nLet\u0026rsquo;s see how the change of interpolation changed the maximum speed estimate:\nmax_speed_2008_s5degree = Q_(max(s_degree5_2008_beijing(time, 1)), \u0026#39;m/s\u0026#39;) max_speed_2009_s5degree = Q_(max(s_degree5_2009_berlin(time, 1)), \u0026#39;m/s\u0026#39;) print(f\u0026#39;{max_speed_2008_s5degree:.2f}\u0026#39;) print(f\u0026#39;{max_speed_2009_s5degree:.2f}\u0026#39;) 12.28 meters per second 12.27 meters per second Since the curves were \u0026ldquo;smoothed\u0026rdquo;, there are no longer so many maxima and minima in the plot. Did Bolt really reach a lower maximum speed in 2009 compared to 2008?\nAre our models right? # No. Models try to describe reality, they are not reality. It does not make much sense to say \u0026ldquo;right\u0026rdquo; or \u0026ldquo;wrong\u0026rdquo;, we can evaluate, based on the data we have, whether they are coherent and whether they make predictions that make sense. That is, the discussions we have been having at each step in this article. They can be, at most, as good as the input data, the famous GIGO - garbage in, garbage out. Since we only have position data every 10 meters, everything that comes out of this analysis will be limited by that input data.\nIn the case of the maximum speed, for example, if it was reached at some intermediate point, obviously the form of interpolation used to obtain those points will have a great effect on the result.\nGetting experimental speed data # Bolt\u0026rsquo;s performance at the 2008 Olympics, when he broke the 100 m world record for the first time, drew so much attention that it changed how the scientific community prepared for the 2009 World Championships. Many wondered what the time would be if he had not slowed down to celebrate in the final meters, and there was great interest in understanding the Jamaican\u0026rsquo;s performance.\nSo, special cameras, speed meters and many other pieces of equipment were used to study the biomechanics of the athletes in the competition. The resulting report is well worth reading. The details are amazing.\nAn interesting point of the report is that, with the speed meters used, they were able to estimate the athletes\u0026rsquo; average speeds every 10 meters. Let\u0026rsquo;s import this data and compare it with our models:\n!cat bolt_berlin_vel.csv Splits,\u0026quot;2009 Berlin\u0026quot; 0,0 \u0026quot;0 - 10\u0026quot;,5.77 \u0026quot;10 - 20\u0026quot;,9.99 \u0026quot;20 - 30\u0026quot;,11.11 \u0026quot;30 - 40\u0026quot;,11.63 \u0026quot;40 - 50\u0026quot;,12.08 \u0026quot;50 - 60\u0026quot;,12.20 \u0026quot;60 - 70\u0026quot;,12.29 \u0026quot;70 - 80\u0026quot;,12.17 \u0026quot;80 - 90\u0026quot;,12.17 \u0026quot;90 - 100\u0026quot;,11.96 berlin_speed_file = \u0026#39;bolt_berlin_vel.csv\u0026#39; berlin_speed_data = np.genfromtxt(berlin_speed_file, delimiter=\u0026#39;,\u0026#39;, dtype=[\u0026#39;U10\u0026#39;, float], names=True) berlin_speed_data array([('0', 0. ), ('\u0026quot;0 - 10\u0026quot;', 5.77), ('\u0026quot;10 - 20\u0026quot;', 9.99), ('\u0026quot;20 - 30\u0026quot;', 11.11), ('\u0026quot;30 - 40\u0026quot;', 11.63), ('\u0026quot;40 - 50\u0026quot;', 12.08), ('\u0026quot;50 - 60\u0026quot;', 12.2 ), ('\u0026quot;60 - 70\u0026quot;', 12.29), ('\u0026quot;70 - 80\u0026quot;', 12.17), ('\u0026quot;80 - 90\u0026quot;', 12.17), ('\u0026quot;90 - 100\u0026quot;', 11.96)], dtype=[('Splits', '\u0026lt;U10'), ('2009_Berlin', '\u0026lt;f8')]) The maximum speed obtained was:\nQ_(max(berlin_speed_data[\u0026#39;2009_Berlin\u0026#39;]), \u0026#39;m/s\u0026#39;) 12.29 meters per second But remember, this speed is the result of an average over each 10 meters. I will explain more about this shortly. Bolt\u0026rsquo;s real maximum speed in the race, according to the cited report, was 12.34 m/s when he had already covered 67.90 m.\nLet\u0026rsquo;s compare this data with our model made with the cubic spline:\nplt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Speed / m/s\u0026#39;) plt.title(\u0026#39;Usain Bolt - Berlin World Championships - 2009\u0026#39;) plt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], berlin_speed_data[\u0026#39;2009_Berlin\u0026#39;], label=\u0026#39;Experimental mean speed\u0026#39;) plt.plot(time, cs_2009_berlin(time, 1), label=\u0026#39;Cubic spline\u0026#39;) plt.legend() plt.show() And now with the fifth-degree spline:\nplt.xlabel(\u0026#39;Time / s\u0026#39;) plt.ylabel(\u0026#39;Speed / m/s\u0026#39;) plt.title(\u0026#39;Usain Bolt - Berlin World Championships - 2009\u0026#39;) plt.scatter(data[\u0026#39;2009_Berlin\u0026#39;], berlin_speed_data[\u0026#39;2009_Berlin\u0026#39;], label=\u0026#39;Experimental mean speed\u0026#39;) plt.plot(time, s_degree5_2009_berlin(time, 1), label=\u0026#39;degree 5 spline\u0026#39;) plt.legend() plt.show() So? Any conclusion? Hard to say. Both fail in the initial times. The fifth-degree model better shows the drop in speed in the final stretch. But we must be careful about whether the comparison makes sense.\nThe interpolation model generates intermediate points in the position data and creates the position spline, which is then differentiated to obtain the speed curve. The speed tabulated in the report is, as already said, an average over each 10-meter interval based on the speed measurements. Let\u0026rsquo;s look at one of the plots in the report to understand:\nUnfortunately the resolution of the available figure is not the best. But we have speed on the vertical axis and position on the horizontal one. The blue line is the result of the experimental data from the meters and the red one is an average line of that data.\nCan you understand why they used an average? The speed oscillates a lot, after all, when running there is the rhythmic impact of each foot, in each stride. In fact, the report itself and other references like this and this show that Bolt\u0026rsquo;s great edge is in his stride. Reading the papers also makes clear the experimental difficulties of measuring speed: after all, we are dealing with bodies and not points, so one must determine which position of the body will be tracked, the influence of the athlete\u0026rsquo;s reaction time on the results, especially in the first meters, among other factors.\nWhat could be improved? # I hope you understood that there are many factors involved and that this is just an exercise to demonstrate the Python language, the available mathematical tools and a bit of calculus and physics. But let\u0026rsquo;s end the article by thinking about how we could improve our analysis.\nLook again at the speed plots we obtained. Is there anything strange? Well, I don\u0026rsquo;t know if you noticed, but the plots do not start at zero. This can be solved by forcing a boundary condition where the first derivative must be zero at the initial point. This can be done, and I already mentioned that there are different types of spline depending on the boundary conditions. Reading the documentation of the spline methods shows how this could be done. Write in the comments section if you would like an article where this is done, or if you have done it and want to show it.\nBut it is not that simple, since there is the athlete\u0026rsquo;s reaction time. The speed is zero until the athlete starts moving. It is up to you to consider whether this will affect the analysis much or not. And what it would be like to implement this. Oh, and are you going to try to discount the influence of the wind or not?\nAnother approach is to start from mathematical models previously defined in the literature. I recommend reading this paper and also this one. That way, the boundary conditions could be the equations proposed in these papers and it would be up to us to check whether they represent the situations well or not.\nFinally, remember above where I wrote about the polynomial interpolation method? When you look for a polynomial that passes through all the points? Other people have already done studies along these lines: as here and here. See the results and compare them with the approach we used. Also try to reproduce the studies and draw your own conclusions.\nConclusion # Phew, this article was not easy to write. A lot of reading, math, sports, physics and Python. But the goal was to show the mathematical tools available for the language using a real problem. I believe that is much more productive than showing the tired examples that every book and site has.\nAnd using a real problem also lets us discuss the limitations of methods and models and how to analyze them critically. Unfortunately, I see some problems in the way they are taught and used today. Namely:\nmany confuse model with reality; tools are applied in contexts where they do not apply; related to the previous item, there are many users of methods and models, but few analysts and people who understand them; real cases are rarely used in teaching, so critical analysis is not exercised. Textbook data is always so pretty, right? the media\u0026rsquo;s popularization/interpretation of scientific studies and of model results is deplorable. The consequence of all this relates to what we are living through in this year of 2020, when I write this article. After all, a good part of what we lived through this year was a consequence of predictions made from models at the beginning of the year. Remember (in Portuguese)? Obviously the disease is serious and must not be neglected, that is not the point. But was every precaution taken in the model and in its software? It seems not (in Portuguese). Did they think about the political consequences and about people\u0026rsquo;s lives and freedom? Did the predictions come true? Food for thought.\nSee you next time.\n","date":"2020-09-10","externalUrl":null,"permalink":"/2020/09/bolt-python-olympic-athlete-speed/","section":"Blog","summary":"Usain Bolt is considered by many to be the greatest sprinter of all time. In this article, we will see how to use Python to find the speed and acceleration the Jamaican reached in his races. We will also take the chance to discuss mathematical models and the care needed in choosing them and interpreting their results.","title":"Bolt vs Python: finding the speed of an Olympic athlete","type":"posts"},{"content":" In the last article on the site, I showed the incredible pint library for handling units, and a way to computationally solve the conversion problem that left an airplane without fuel in mid-flight, as described in that other article.\nToday, we will keep talking about units with the pint library, showing how it integrates with the numpy library and with plots made with matplotlib.\nFirst, let\u0026rsquo;s import the necessary packages:\nimport pint import numpy as np import matplotlib.pyplot as plt Now, create the unit registry and the quantity constructor of pint:\nureg = pint.UnitRegistry() Q_ = ureg.Quantity Let\u0026rsquo;s describe a motion # As an example, I decided to take a high school physics problem. I picked up the book I used in high school and found an exercise on graphs of uniformly accelerated motion (UAM).\nThe book is \u0026ldquo;Fundamentos de Física - Volume 1: Mecânica\u0026rdquo; by Francisco Ramalho Júnior and others, Ed. Moderna, 1976.\nRelax, if you don\u0026rsquo;t remember what UAM is, here is a quick recap (read it to the sound of Telecurso 2000, in Portuguese):\nMotions are classified into two broad categories: uniform motions, which have constant velocity, and accelerated motions, those whose velocity varies in time. In uniform motion, the average velocity computed over any time interval is always the same, which does not happen in accelerated motion. Uniformly accelerated motion (UAM) is a motion in which the velocity varies uniformly with time. Acceleration is measured by the change of velocity in time. In UAM the acceleration is constant over time. A UAM has scalar acceleration constant in time and a velocity that varies according to the function $$ v = v_0 + \\alpha t $$ where \\(v_0\\) is the initial velocity, \\(\\alpha\\) is the acceleration and \\(t\\) is time. For the description of the motion to be complete, we must know how the position \\(x\\) varies in time: $$ x(t) = x_0 + v_0 t + \\frac{1}{2} \\alpha t^2 $$Now that we have reviewed, let\u0026rsquo;s define what we want:\nwe want to build a plot of position as a function of time, that is, \\(x(t)\\); therefore, we need points for this plot in the (time, position) format; the points assume that position is in meters and time in seconds; to have a minimally acceptable plot, we need a few points. Since we are using a computer and have a library like numpy available to do computations quickly, we will consider an acceptable minimum of 50 points, and the user may select more or fewer points if desired; we will use the fact that we can define the position at any time \\(t\\) in a UAM knowing only the initial position, the initial velocity and the acceleration; we know that a Python function can return more than one value and, when it does, it returns the values as a tuple; we will use the data from the book exercise as a test for the function to be defined; by default, the function takes the initial time to be zero, leaving it up to the user to pass the final time. Let\u0026rsquo;s split the problem into two parts:\nObtaining the (time, position) points Plotting the graph We can then sketch our desired function, which we will call uniformly_accelerated_motion, as:\n# SKETCH def uniformly_accelerated_motion(initial_position, initial_velocity, acceleration, final_time, initial_time=Q_(\u0026#39;0 s\u0026#39;), points=50): # define the array of time values with numpy linspace # compute the position at each time, generating an array of position values # attach units to the values return time, position The linspace method of numpy is excellent for obtaining equally spaced values, in this case time values, generating an array (vector). Then it is enough to take that array and compute the position at each time. Using the wraps of pint, explained in the last article, we can define each desired unit. Implementing the function then:\n@ureg.wraps((\u0026#39;second\u0026#39;, \u0026#39;meter\u0026#39;), (\u0026#39;meter\u0026#39;, \u0026#39;m/s\u0026#39;, \u0026#39;m/(s**2)\u0026#39;, \u0026#39;second\u0026#39;, \u0026#39;second\u0026#39;, None)) def uniformly_accelerated_motion(initial_position, initial_velocity, acceleration, final_time, initial_time=Q_(\u0026#39;0 s\u0026#39;), points=50): \u0026#39;\u0026#39;\u0026#39; Returns (time, position) pairs for an equation that describes the position in a uniformly accelerated motion Parameters: ----------- initial_position: pint.Quantity, a quantity in meters is expected initial_velocity: pint.Quantity, a quantity in meters per second is expected acceleration: pint.Quantity, a quantity in meters per second squared is expected final_time: pint.Quantity, a quantity in seconds is expected initial_time: pint.Quantity, a quantity in seconds is expected, default 0 points: integer, number of points to be computed, default 50 Returns: -------- tuple, 2 arrays for (time, position) with units via pint \u0026#39;\u0026#39;\u0026#39; time = np.linspace(initial_time, final_time, points) position = initial_position + initial_velocity * time + 1/2 * acceleration * time**2 return time, position Now let\u0026rsquo;s create the variables that will be passed as parameters of the function, with the values from the exercise:\ninit_pos = Q_(\u0026#39;12 m\u0026#39;) init_vel = Q_(\u0026#39;-8 m/s\u0026#39;) accel = Q_(\u0026#39;2 m/(s**2)\u0026#39;) final_t = Q_(\u0026#39;8 s\u0026#39;) uam = uniformly_accelerated_motion(init_pos, init_vel, accel, final_t) Let\u0026rsquo;s check what the uam variable looks like:\nuam (array([0. , 0.16326531, 0.32653061, 0.48979592, 0.65306122, 0.81632653, 0.97959184, 1.14285714, 1.30612245, 1.46938776, 1.63265306, 1.79591837, 1.95918367, 2.12244898, 2.28571429, 2.44897959, 2.6122449 , 2.7755102 , 2.93877551, 3.10204082, 3.26530612, 3.42857143, 3.59183673, 3.75510204, 3.91836735, 4.08163265, 4.24489796, 4.40816327, 4.57142857, 4.73469388, 4.89795918, 5.06122449, 5.2244898 , 5.3877551 , 5.55102041, 5.71428571, 5.87755102, 6.04081633, 6.20408163, 6.36734694, 6.53061224, 6.69387755, 6.85714286, 7.02040816, 7.18367347, 7.34693878, 7.51020408, 7.67346939, 7.83673469, 8. ]) \u0026lt;Unit('second')\u0026gt;, array([12. , 10.72053311, 9.49437734, 8.32153269, 7.20199917, 6.13577676, 5.12286547, 4.16326531, 3.25697626, 2.40399833, 1.60433153, 0.85797584, 0.16493128, -0.47480217, -1.06122449, -1.59433569, -2.07413578, -2.50062474, -2.87380258, -3.1936693 , -3.46022491, -3.67346939, -3.83340275, -3.94002499, -3.99333611, -3.99333611, -3.94002499, -3.83340275, -3.67346939, -3.46022491, -3.1936693 , -2.87380258, -2.50062474, -2.07413578, -1.59433569, -1.06122449, -0.47480217, 0.16493128, 0.85797584, 1.60433153, 2.40399833, 3.25697626, 4.16326531, 5.12286547, 6.13577676, 7.20199917, 8.32153269, 9.49437734, 10.72053311, 12. ]) \u0026lt;Unit('meter')\u0026gt;) Notice that it is a tuple of two arrays, one for time and another for position, and that these arrays have associated units, as desired through the use of the pint package.\nIntegration between pint and matplotlib # To plot, there is no need to create a function. Just pass the values of the horizontal and vertical axes to the plot method of matplotlib.pyplot:\nplt.plot(uam[0], uam[1]) /home/chicolucio/Dropbox/work/cienciaprogramada/code/pint/.venv/lib/python3.8/site-packages/numpy/core/_asarray.py:83: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray. return array(a, dtype, copy=False, order=order) Note that a warning was displayed, informing that the units were stripped. This happens because we have not configured the unit registry to work with matplotlib. To configure it, just use the following command:\nureg.setup_matplotlib(True) Requesting the plot again:\nplt.plot(uam[0], uam[1]) The warning is gone. But there is still a lot to improve.\nImproving code readability # I try, within the limits of my knowledge of the language of course, to always follow the Zen of Python:\nimport this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! Especially Readability counts. That is why I defined the function with parameters whose names actually mean something (and not that madness of x, v, t and the like, that works on paper, not in code). And it really bothers me to have to call tuples by indices. It would be better to be able to do something like uam.time and uam.position. And in fact that is possible\u0026hellip;\nNamed tuples, namedtuple, are made for this:\nfrom collections import namedtuple Slightly adapting the function defined previously:\n@ureg.wraps((\u0026#39;second\u0026#39;, \u0026#39;meter\u0026#39;), (\u0026#39;meter\u0026#39;, \u0026#39;m/s\u0026#39;, \u0026#39;m/(s**2)\u0026#39;, \u0026#39;second\u0026#39;, \u0026#39;second\u0026#39;, None)) def uniformly_accelerated_motion(initial_position, initial_velocity, acceleration, final_time, initial_time=Q_(\u0026#39;0 s\u0026#39;), points=50): \u0026#39;\u0026#39;\u0026#39; Returns (time, position) pairs for an equation that describes the position in a uniformly accelerated motion Parameters: ----------- initial_position: pint.Quantity, a quantity in meters is expected initial_velocity: pint.Quantity, a quantity in meters per second is expected acceleration: pint.Quantity, a quantity in meters per second squared is expected final_time: pint.Quantity, a quantity in seconds is expected initial_time: pint.Quantity, a quantity in seconds is expected, default 0 points: integer, number of points to be computed, default 50 Returns: -------- named tuple, 2 arrays for (time, position) with units via pint \u0026#39;\u0026#39;\u0026#39; time = np.linspace(initial_time, final_time, points) position = initial_position + initial_velocity * time + 1/2 * acceleration * time**2 Data = namedtuple(\u0026#39;UAM\u0026#39;, [\u0026#39;time\u0026#39;, \u0026#39;position\u0026#39;]) return Data(time, position) uam = uniformly_accelerated_motion(init_pos, init_vel, accel, final_t) uam UAM(time=\u0026lt;Quantity([0. 0.16326531 0.32653061 0.48979592 0.65306122 0.81632653 0.97959184 1.14285714 1.30612245 1.46938776 1.63265306 1.79591837 1.95918367 2.12244898 2.28571429 2.44897959 2.6122449 2.7755102 2.93877551 3.10204082 3.26530612 3.42857143 3.59183673 3.75510204 3.91836735 4.08163265 4.24489796 4.40816327 4.57142857 4.73469388 4.89795918 5.06122449 5.2244898 5.3877551 5.55102041 5.71428571 5.87755102 6.04081633 6.20408163 6.36734694 6.53061224 6.69387755 6.85714286 7.02040816 7.18367347 7.34693878 7.51020408 7.67346939 7.83673469 8. ], 'second')\u0026gt;, position=\u0026lt;Quantity([12. 10.72053311 9.49437734 8.32153269 7.20199917 6.13577676 5.12286547 4.16326531 3.25697626 2.40399833 1.60433153 0.85797584 0.16493128 -0.47480217 -1.06122449 -1.59433569 -2.07413578 -2.50062474 -2.87380258 -3.1936693 -3.46022491 -3.67346939 -3.83340275 -3.94002499 -3.99333611 -3.99333611 -3.94002499 -3.83340275 -3.67346939 -3.46022491 -3.1936693 -2.87380258 -2.50062474 -2.07413578 -1.59433569 -1.06122449 -0.47480217 0.16493128 0.85797584 1.60433153 2.40399833 3.25697626 4.16326531 5.12286547 6.13577676 7.20199917 8.32153269 9.49437734 10.72053311 12. ], 'meter')\u0026gt;) Now we can call them by name:\nuam.time Magnitude[0.0 0.16326530612244897 0.32653061224489793 0.4897959183673469\n0.6530612244897959 0.8163265306122448 0.9795918367346939\n1.1428571428571428 1.3061224489795917 1.4693877551020407\n1.6326530612244896 1.7959183673469385 1.9591836734693877\n2.1224489795918364 2.2857142857142856 2.4489795918367343\n2.6122448979591835 2.7755102040816326 2.9387755102040813\n3.1020408163265305 3.265306122448979 3.4285714285714284 3.591836734693877\n3.7551020408163263 3.9183673469387754 4.081632653061225 4.244897959183673\n4.408163265306122 4.571428571428571 4.73469387755102 4.897959183673469\n5.061224489795918 5.224489795918367 5.387755102040816 5.551020408163265\n5.7142857142857135 5.877551020408163 6.040816326530612 6.204081632653061\n6.367346938775509 6.530612244897958 6.693877551020408 6.857142857142857\n7.020408163265306 7.183673469387754 7.346938775510203 7.5102040816326525\n7.673469387755102 7.836734693877551 8.0]Unitssecond uam.position Magnitude[12.0 10.720533111203665 9.494377342773845 8.321532694710536\n7.201999167013745 6.135776759683466 5.122865472719701 4.163265306122449\n3.2569762598917125 2.4039983340274893 1.6043315285297801\n0.8579758433985849 0.16493127863390278 -0.47480216576426404\n-1.0612244897959178 -1.5943356934610566 -2.0741357767596833\n-2.500624739691795 -2.873802582257392 -3.193669304456476\n-3.4602249062890458 -3.6734693877551017 -3.8334027488546436\n-3.9400249895876716 -3.9933361099541855 -3.9933361099541855\n-3.9400249895876733 -3.8334027488546454 -3.6734693877551017\n-3.4602249062890458 -3.1936693044564777 -2.873802582257394\n-2.5006247396917978 -2.0741357767596824 -1.5943356934610584\n-1.0612244897959187 -0.4748021657642667 0.164931278633901\n0.8579758433985845 1.6043315285297766 2.4039983340274844\n3.256976259891708 4.163265306122447 5.122865472719695 6.135776759683459\n7.201999167013739 8.321532694710534 9.494377342773845 10.720533111203665\n12.0]Unitsmeter But the interesting part is that the logic of calling by indices still works:\nuam[0] Magnitude[0.0 0.16326530612244897 0.32653061224489793 0.4897959183673469\n0.6530612244897959 0.8163265306122448 0.9795918367346939\n1.1428571428571428 1.3061224489795917 1.4693877551020407\n1.6326530612244896 1.7959183673469385 1.9591836734693877\n2.1224489795918364 2.2857142857142856 2.4489795918367343\n2.6122448979591835 2.7755102040816326 2.9387755102040813\n3.1020408163265305 3.265306122448979 3.4285714285714284 3.591836734693877\n3.7551020408163263 3.9183673469387754 4.081632653061225 4.244897959183673\n4.408163265306122 4.571428571428571 4.73469387755102 4.897959183673469\n5.061224489795918 5.224489795918367 5.387755102040816 5.551020408163265\n5.7142857142857135 5.877551020408163 6.040816326530612 6.204081632653061\n6.367346938775509 6.530612244897958 6.693877551020408 6.857142857142857\n7.020408163265306 7.183673469387754 7.346938775510203 7.5102040816326525\n7.673469387755102 7.836734693877551 8.0]Unitssecond uam[1] Magnitude[12.0 10.720533111203665 9.494377342773845 8.321532694710536\n7.201999167013745 6.135776759683466 5.122865472719701 4.163265306122449\n3.2569762598917125 2.4039983340274893 1.6043315285297801\n0.8579758433985849 0.16493127863390278 -0.47480216576426404\n-1.0612244897959178 -1.5943356934610566 -2.0741357767596833\n-2.500624739691795 -2.873802582257392 -3.193669304456476\n-3.4602249062890458 -3.6734693877551017 -3.8334027488546436\n-3.9400249895876716 -3.9933361099541855 -3.9933361099541855\n-3.9400249895876733 -3.8334027488546454 -3.6734693877551017\n-3.4602249062890458 -3.1936693044564777 -2.873802582257394\n-2.5006247396917978 -2.0741357767596824 -1.5943356934610584\n-1.0612244897959187 -0.4748021657642667 0.164931278633901\n0.8579758433985845 1.6043315285297766 2.4039983340274844\n3.256976259891708 4.163265306122447 5.122865472719695 6.135776759683459\n7.201999167013739 8.321532694710534 9.494377342773845 10.720533111203665\n12.0]Unitsmeter But see whether the following plot command isn\u0026rsquo;t much easier to understand:\nplt.plot(uam.time, uam.position) I don\u0026rsquo;t know about you, but to me this code is much more expressive, it communicates much better what is being requested.\nOf course, the full power of pint can be used in plots as well. If you want a plot with time in milliseconds and position in feet:\nplt.plot(uam.time.to(\u0026#39;ms\u0026#39;), uam.position.to(\u0026#39;ft\u0026#39;)) Exploring the integration with matplotlib further # For the purposes of this article, what has been shown so far would already be enough. But I will take the opportunity to show some slightly more advanced matplotlib things and how to combine them with pint.\nmatplotlib has some predefined plot styles that can be seen here. But I particularly like making my own styles, since I have some more specific needs. For example, since I work in education, I like plots to show grid lines at the major and minor divisions, because this makes reading easier for the student, whether in reference material or in an exercise.\nA good way to have standardized plots throughout a document is to have a function that defines the grid, tick and font parameters for a given axis, and to call that function in the body of each plot function used in the document. After all, there may be more than one plot function, since there may be plots that do not need, for example, unit handling, or plots of different mathematical functions (in the context of this article, uniform or accelerated motions).\nSo, let\u0026rsquo;s define a function that returns axes with the settings I want. I still plan to write a more detailed article about matplotlib terminology, but in case you don\u0026rsquo;t know the difference between axes and figure, I recommend this link in the documentation and this one from StackOverflow.\ndef _plot_params(ax=None): \u0026#34;\u0026#34;\u0026#34;Custom parameters for plots. Parameters: ---------- ax : Matplotlib axes, optional, axes where the plot will be drawn, default None Returns: -------- Matplotlib axes, axes for a figure \u0026#34;\u0026#34;\u0026#34; linewidth = 2 size = 12 # grid and ticks ax.minorticks_on() ax.grid(b=True, which=\u0026#39;major\u0026#39;, linestyle=\u0026#39;--\u0026#39;, linewidth=linewidth - 0.5) ax.grid(b=True, which=\u0026#39;minor\u0026#39;, axis=\u0026#39;both\u0026#39;, linestyle=\u0026#39;:\u0026#39;, linewidth=linewidth - 1) ax.tick_params(which=\u0026#39;both\u0026#39;, labelsize=size+2) ax.tick_params(which=\u0026#39;major\u0026#39;, length=6, axis=\u0026#39;both\u0026#39;) ax.tick_params(which=\u0026#39;minor\u0026#39;, length=3, axis=\u0026#39;both\u0026#39;) # axis names ax.xaxis.label.set_size(size+4) ax.yaxis.label.set_size(size+4) return Now let\u0026rsquo;s define a function to plot our UAM data. A way of working that I usually adopt is to always work with matplotlib axes, because this gives me more flexibility if I want, for example, to make figures with side-by-side plots. I will illustrate this later, in case it wasn\u0026rsquo;t clear. But that is why the function has an ax parameter. This parameter is, by default, None and, if an axes is really not passed to the function, a figure with one axes will be created (the if conditional clause in the function):\ndef plot_uam(time_array, position_array, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;m\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, ax=None, fig_size=(8, 6)): \u0026#39;\u0026#39;\u0026#39; Function to plot the graph of a Uniformly Accelerated Motion Parameters: ---------- time_array: pint array, array of time-related values with an associated unit position_array: pint array, array of position-related values with an associated unit time_unit: string, unit to use with the time values, default \u0026#34;seconds\u0026#34; position_unit: string, unit to use with the position values, default \u0026#34;meters\u0026#34; y_axis_name: string, name for the vertical axis, default \u0026#39;Position\u0026#39; x_axis_name: string, name for the horizontal axis, default \u0026#39;Time\u0026#39; ax: Matplotlib axes, axes where the plot will be drawn, default None fig_size: tuple, size of the figure that will be created if no axes is passed, default (8, 6) Returns: -------- Matplotlib axes, axes for a figure \u0026#39;\u0026#39;\u0026#39; if ax is None: fig, ax = plt.subplots(figsize=fig_size, facecolor=(1.0, 1.0, 1.0)) # calling the function for the custom parameters _plot_params(ax) # plot, converting the data to the units passed ax.plot(time_array.to(time_unit), position_array.to(position_unit)) # customizing the axis names with the standard adopted by the scientific community (Axis name / unit) ax.set_xlabel(x_axis_name + \u0026#39; / \u0026#39; + f\u0026#39;{ureg(time_unit).units:~P}\u0026#39;) ax.set_ylabel(y_axis_name + \u0026#39; / \u0026#39; + f\u0026#39;{ureg(position_unit).units:~P}\u0026#39;) Let\u0026rsquo;s check what the plot looks like with the data we already have:\nplot_uam(uam.time, uam.position) Much better, don\u0026rsquo;t you think?\nLet\u0026rsquo;s customize it with other units, changing the axis names and the size:\nplot_uam(uam.time, uam.position, time_unit=\u0026#39;ms\u0026#39;, position_unit=\u0026#39;ft\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, fig_size=(7, 5)) Now let\u0026rsquo;s check the power of working with axes in matplotlib. We can create a grid of plots, each with its own system of units. I will use the subplots method, whose documentation is here.\n# creating a grid of plots with 2 columns and 2 rows fig, arr = plt.subplots(nrows=2, ncols=2, figsize=(14,10), constrained_layout=True, facecolor=(1,1,1)) # arr is a matrix of positions for the plots. See the documentation link above plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;m\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[0, 0]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;ms\u0026#39;, position_unit=\u0026#39;ft\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[0, 1]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;yards\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[1, 0]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;cm\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[1, 1]) Notice what we just did. It is the same data set, passed only once, and four plots were generated with a few lines of code. Conversions become very simple with pint.\nWe can elaborate our function even further to show the axes in scientific notation and modify the axis name to display the order of magnitude (the power of 10 of the notation).\ndef plot_uam(time_array, position_array, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;m\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, ax=None, fig_size=(8, 6)): \u0026#39;\u0026#39;\u0026#39; Function to plot the graph of a Uniformly Accelerated Motion Parameters: ---------- time_array: array, array of time-related values with an associated unit position_array: array, array of position-related values with an associated unit time_unit: string, unit to use with the time values, default seconds position_unit: string, unit to use with the position values, default meters y_axis_name: string, name for the vertical axis, allowing translation to other languages, default \u0026#39;Position\u0026#39; x_axis_name: string, name for the horizontal axis, allowing translation to other languages, default \u0026#39;Time\u0026#39; ax: Matplotlib axes, axes where the plot will be drawn, default None fig_size: tuple, size of the figure that will be created if no axes is passed, default (8, 6) Returns: -------- Matplotlib axes, axes for a figure \u0026#39;\u0026#39;\u0026#39; if ax is None: fig, ax = plt.subplots(figsize=fig_size, facecolor=(1.0, 1.0, 1.0)) # calling the function for the custom parameters _plot_params(ax) # plot, converting the data to the units passed ax.plot(time_array.to(time_unit), position_array.to(position_unit)) # formats the axes in scientific notation style, using a power of 10 instead of \u0026#34;E\u0026#34; ax.ticklabel_format(style=\u0026#39;sci\u0026#39;, axis=\u0026#39;both\u0026#39;, scilimits=(0, 0)) ax.yaxis.major.formatter._useMathText = True ax.xaxis.major.formatter._useMathText = True ax.figure.canvas.draw() # needs a redraw to take effect # removes the multiplication sign from the power of 10, storing it in order-of-magnitude variables order_magnitude_y = ax.yaxis.get_offset_text().get_text().replace(\u0026#39;\\\\times\u0026#39;, \u0026#39;\u0026#39;) order_magnitude_x = ax.xaxis.get_offset_text().get_text().replace(\u0026#39;\\\\times\u0026#39;, \u0026#39;\u0026#39;) # turns off matplotlib\u0026#39;s default display so we can show our custom one ax.yaxis.offsetText.set_visible(False) ax.xaxis.offsetText.set_visible(False) # customizing the axis names with the standard adopted by the scientific community (Axis name / unit) ax.set_xlabel(x_axis_name + \u0026#39; / \u0026#39; + order_magnitude_x + f\u0026#39; {ureg(time_unit).units:~P}\u0026#39;) ax.set_ylabel(y_axis_name + \u0026#39; / \u0026#39; + order_magnitude_y + f\u0026#39; {ureg(position_unit).units:~P}\u0026#39;) Let\u0026rsquo;s test this new implementation:\nplot_uam(uam.time, uam.position) Now the position axis is in scientific notation, making that clear in the axis name. The time axis does not need a power of 10, so it is not displayed. Let\u0026rsquo;s test with other units:\nplot_uam(uam.time, uam.position, time_unit=\u0026#39;ms\u0026#39;, position_unit=\u0026#39;ft\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;) Notice that both axes are in scientific notation and display this information in the axis names.\nCreating our grid of plots again:\n# creating a grid of plots with 2 columns and 2 rows fig, arr = plt.subplots(nrows=2, ncols=2, figsize=(14,10), constrained_layout=True, facecolor=(1,1,1)) # arr is a matrix of positions for the plots. See the documentation link above plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;m\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[0, 0]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;ms\u0026#39;, position_unit=\u0026#39;ft\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[0, 1]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;yards\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[1, 0]) plot_uam(uam.time, uam.position, time_unit=\u0026#39;s\u0026#39;, position_unit=\u0026#39;cm\u0026#39;, x_axis_name=\u0026#39;Time\u0026#39;, y_axis_name=\u0026#39;Position\u0026#39;, ax=arr[1, 1]) Conclusion # The combination of pint, numpy and matplotlib is sensational. With a few lines of code we can attach units to data sets and create different plots for different unit systems using the same data set.\nThere are no more excuses for not using this combination in code involving physical quantities, right? And there is more to come, this trio of libraries will be very present here in the articles.\nSee you next time!\n","date":"2020-09-05","externalUrl":null,"permalink":"/2020/09/python-plots-pint-matplotlib/","section":"Blog","summary":"In this article we detail how to get visually elegant plots that correctly show units on their axes. We will combine the pint, numpy and matplotlib libraries.","title":"Plots with units in Python: pint and matplotlib","type":"posts"},{"content":"","date":"2020-09-05","externalUrl":null,"permalink":"/tags/unidades/","section":"tags","summary":"","title":"unidades","type":"tags"},{"content":"","date":"2020-09-05","externalUrl":null,"permalink":"/tags/units/","section":"tags","summary":"","title":"units","type":"tags"},{"content":" Whenever we work with measurements and experimental data, we need to express our values with their respective units. In this article, we will see a Python library for working with units and what that has to do with beer.\nIf you prefer to watch a video (in Portuguese), click the player below. The full article follows the video.\nThe pint package # The pint package lets you work with physical quantities: the product of a numerical value and a unit of measure. It has an extensive list of units, prefixes, and constants that can also be extended by the user. It is compatible with several operations of the widely used numpy package.\nIf you are curious about why the symbol on the project website is a beer glass, read this Wikipedia article. The name is nothing more than a reference to a traditional beer glass, very common in European countries and in the United States. And, of course, each country has its own volume for 1 pint, as shown in this table. A great name for a unit conversion package, right?\npint requires Python 3.6+ and has no other dependencies.\nInstallation is quite simple, as shown in the documentation, and can be done via conda or pip:\npip install pint See this article if you want to install it in virtual environments.\npint works with the concept of a unit registry. After importing the package, you create a registry that will store and evaluate the units of the whole code:\nimport pint ureg = pint.UnitRegistry() Of course, this registry, which is just a variable, can have any name, but the usual convention, following the documentation, is to call it ureg.\nWe can now check that the package will handle operations involving units:\n1 * ureg.meter + 100 * ureg.cm 2.0 meter Notice that in the sum above, the package recognized that 100 cm is equivalent to 1 m and returned the result as 2 m. Let us go through the use of the package slowly to understand this behavior.\nCreating quantities # Let us consider a simple situation, where we want to calculate an average speed, given a change in distance over a given time interval. We can create a variable with magnitude and unit in two ways. The first is through the concept that a quantity is a numerical value multiplied by a unit:\ndistance = 50 * ureg.meter distance 50 meter The second is via a quantity constructor. Again, the constructor can have any name, but the convention created by the package documentation is to call it Q_:\nQ_ = ureg.Quantity time = Q_(10, ureg.second) time 10 second To calculate the average speed, just follow the physical definition:\n\\[\\overline{v} = \\frac{\\Delta x}{\\Delta t} \\]\nwhere \\(\\overline{v}\\) is the average speed, \\(x\\) is the distance, \\(t\\) is the time, and the symbol \\(\\Delta\\) indicates change.\naverage_speed = distance / time average_speed 5.0 meter/second Notice that the unit of the speed is correct, considering the units passed to the variables related to distance and time.\nWe can check that, regardless of how the variables were defined, they are all of type Quantity:\nprint(type(distance)) print(type(time)) print(type(average_speed)) \u0026lt;class 'pint.quantity.build_quantity_class.\u0026lt;locals\u0026gt;.Quantity'\u0026gt; \u0026lt;class 'pint.quantity.build_quantity_class.\u0026lt;locals\u0026gt;.Quantity'\u0026gt; \u0026lt;class 'pint.quantity.build_quantity_class.\u0026lt;locals\u0026gt;.Quantity'\u0026gt; This can also be verified by checking the repr of each of these objects:\nprint(repr(distance)) print(repr(time)) print(repr(average_speed)) \u0026lt;Quantity(50, 'meter')\u0026gt; \u0026lt;Quantity(10, 'second')\u0026gt; \u0026lt;Quantity(5.0, 'meter / second')\u0026gt; An interesting thing we can notice from the repr is that the unit can also be passed as a string. And here we have a great power of pint. See how we could declare the distance in several ways:\n10 * ureg.meter 10 meter 10 * ureg.m 10 meter 10 * ureg(\u0026#39;meter\u0026#39;) 10 meter 10 * ureg(\u0026#39;m\u0026#39;) 10 meter Q_(10, ureg.meter) 10 meter Q_(\u0026#39;10 m\u0026#39;) 10 meter Notice that the last form is the way we usually describe a quantity, by hand and in written work. This is a great advantage of pint: the package allows parsing (transforming) strings, so I consider this the easiest form. For example, the universal gas constant, in SI units and rounded to 3 decimal places, could be declared as:\nQ_(\u0026#39;8.314 J/(mol*K)\u0026#39;) 8.314 joule/(kelvin mole) Notice the care with parentheses. Units also follow precedence rules in the usual way.\nConverting units # One way to convert is by using the to method:\naverage_speed.to(ureg.km / ureg.hour) 18.0 kilometer/hour Since we have already seen that the package accepts strings:\naverage_speed.to(\u0026#39;km/hour\u0026#39;) # kilometers per hour 18.0 kilometer/hour average_speed.to(\u0026#39;ft/s\u0026#39;) # feet per second 16.404199475065617 foot/second When using to, the quantity bound to the variable keeps its original units:\naverage_speed 5.0 meter/second For the conversion to be made permanent, we must use the ito method (the i can be understood as inplace):\naverage_speed.ito(\u0026#39;km/hour\u0026#39;) average_speed 18.0 kilometer/hour pint has the to_base_units method (and its analog ito_base_units) to convert a given quantity to the units of the system of units defined in the unit registry. By default, the unit registry uses the International System of Units (SI). We can check all the available systems:\ndir(ureg.sys) ['Planck', 'SI', 'US', 'atomic', 'cgs', 'imperial', 'mks'] Since the default system is SI, we can take the average speed, currently stored in kilometers per hour, and request the conversion to SI units:\naverage_speed.to_base_units() 5.0 meter/second Considering a context, for example, of being in the Imperial System, we can define that system:\nureg.default_system = \u0026#39;imperial\u0026#39; Now, the average speed can be converted to the units of the imperial system (yards/second):\naverage_speed.to_base_units() 5.46806649168854 yard/second Let us go back to SI:\nureg.default_system = \u0026#39;SI\u0026#39; average_speed.to_base_units() 5.0 meter/second Let us permanently convert back to SI units:\naverage_speed.ito_base_units() average_speed 5.0 meter/second Getting to know each part of the quantity # As we have already described, a physical quantity is defined by a value, the magnitude, and its respective unit. We can check each of these entities:\naverage_speed.magnitude 5.0 average_speed.units meter/second We can also check the dimensions, to do a dimensional analysis:\naverage_speed.dimensionality \u0026lt;UnitsContainer({'[length]': 1, '[time]': -1})\u0026gt; Applying to a real case # In one of the latest articles on the site, we saw the case of an airplane that ran out of fuel during the flight. That situation was caused by a unit conversion error, as described in the post.\nI will take advantage of the case to show how to work with units when defining functions.\nIt is obvious, but worth stressing, that the case is complex and some simplifications will be made to keep the focus on explaining the concepts involved. Those who want more details about measuring fuel density and about fuel consumption in aviation can read here and here (both in Portuguese).\nAs we saw in the article, the density that should have been used in the calculations should be in kg/l. In a real situation, a density table is consulted to find the value to use depending on the temperature (or a density-measuring device is used). Here, we will simplify by assuming that the density is constant, with a value of 0.803 kg/l, the value that should have been used in the case in question:\nfuel_density = Q_(\u0026#39;0.803 kg/l\u0026#39;) fuel_density 0.803 kilogram/liter Recalling the calculations from the article:\nFuel mass already present: \\(7{,}682 \\text{ L} \\times 0.803 \\text{ kg L}^{-1} = 6{,}169 \\text{ kg}\\) Fuel mass to add: \\(22{,}300 \\text{ kg} - 6{,}169 \\text{ kg} = 16{,}131 \\text{ kg}\\) Conversion to volume: \\(\\dfrac{16{,}131 \\text{ kg}}{0.803 \\text{ kg L}^{-1}} = 20{,}088 \\text{ L}\\) So, we can think, at first, of defining a function that will receive the volume of fuel present in the tank and the total mass of fuel needed for the trip. Then it associates these values with units and returns the volume of fuel needed to refuel the airplane. Let us define this function:\ndef refuel_volume_implementation1(current_volume, total_mass, fuel_density=Q_(\u0026#39;0.803 kg/l\u0026#39;)): \u0026#39;\u0026#39;\u0026#39;Returns the volume of fuel that should refuel the airplane. Parameters ---------- current_volume: float, a value in liters is expected total_mass: float, a value in kilograms is expected fuel_density: pint.Quantity, value in kg/l, default of 0.803 kg/l Returns ------- volume to refuel: float, value in liters NOTE: First implementation, not the ideal one. See the most suitable implementation later in the article. \u0026#39;\u0026#39;\u0026#39; current_volume = Q_(current_volume, \u0026#39;l\u0026#39;) total_mass = Q_(total_mass, \u0026#39;kg\u0026#39;) current_mass = current_volume * fuel_density mass_to_add = total_mass - current_mass return mass_to_add / fuel_density refuel_volume_implementation1(7682, 22300) 20088.85927770859 liter Notice that the returned answer is correct. However, the implementation of the function is still not ideal because the function receives only numerical values. Even though the documentation (DOCUMENT your code!) says that the volume and mass values must be in liters and kilograms, respectively, nothing prevents values in other units from being passed by mistake. Someone could, for example, inadvertently pass a volume in cubic meters, and nothing in the code would warn that this is an error.\nLet us reimplement the function:\ndef refuel_volume_implementation2(current_volume, total_mass, fuel_density=Q_(\u0026#39;0.803 kg/l\u0026#39;)): \u0026#39;\u0026#39;\u0026#39;Returns the volume of fuel that should refuel the airplane. Parameters ---------- current_volume: pint.Quantity, a value in liters is expected total_mass: pint.Quantity, a value in kilograms is expected fuel_density: pint.Quantity, by default \u0026#39;0.803 kg/l\u0026#39; Returns ------- volume to refuel: pint.Quantity, value in liters NOTE: Not the ideal implementation. See the most suitable implementation later in the article. \u0026#39;\u0026#39;\u0026#39; if isinstance(current_volume, pint.Quantity) and isinstance(total_mass, pint.Quantity): current_mass = current_volume * fuel_density mass_to_add = total_mass - current_mass return mass_to_add / fuel_density else: raise ValueError(\u0026#39;Provide the values as Pint physical quantities (pint.Quantity)\u0026#39;) In this implementation, I start by checking whether the parameters are of the correct type. This is not very usual in Python, but it serves what we want for now. If the type is not correct, an error is shown to the user.\nLet us now check how the function behaves when passing values without units:\nrefuel_volume_implementation2(7682, 22300) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) \u0026lt;ipython-input-41-d3388580cae6\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 refuel_volume_implementation2(7682, 22300) \u0026lt;ipython-input-40-e5d679f70da5\u0026gt; in refuel_volume_implementation2(current_volume, total_mass, fuel_density) 21 return mass_to_add / fuel_density 22 else: ---\u0026gt; 23 raise ValueError('Provide the values as Pint physical quantities (pint.Quantity)') ValueError: Provide the values as Pint physical quantities (pint.Quantity) Great! Now let us create quantities to pass to the function.\ncurrent_vol = Q_(\u0026#39;7682 l\u0026#39;) total_m = Q_(\u0026#39;22300 kg\u0026#39;) refuel_volume_implementation2(current_vol, total_m) 20088.85927770859 liter Great! Correct answer.\nThere is still a \u0026ldquo;side effect\u0026rdquo; of the implementation as it was done. If we pass the values in other units, pint will not take care of converting them automatically to the units of the measurement system in use. That is, if, for example, we pass the mass in pounds, we will get a somewhat strange answer:\ntotal_m.ito(\u0026#39;lb\u0026#39;) total_m 49163.084467227694 pound refuel_volume_implementation2(current_vol, total_m) 44288.35361077301 liter pound/kilogram Notice that the units pound and kilogram were not unified, even though they are of the same dimension. To solve situations like this, pint has the to_reduced_units method, which combines units that have the same dimensionality. According to the documentation, this feature can be enabled globally but, by default, it is disabled for performance reasons. So let us redefine our function to solve this small problem:\ndef refuel_volume_implementation3(current_volume, total_mass, fuel_density=Q_(\u0026#39;0.803 kg/l\u0026#39;)): \u0026#39;\u0026#39;\u0026#39;Returns the volume of fuel that should refuel the airplane. Parameters ---------- current_volume: pint.Quantity, a value in liters is expected total_mass: pint.Quantity, a value in kilograms is expected fuel_density: pint.Quantity, by default \u0026#39;0.803 kg/l\u0026#39; Returns ------- volume to refuel: pint.Quantity, value in liters NOTE: Not the ideal implementation. See the most suitable implementation later in the article. \u0026#39;\u0026#39;\u0026#39; if isinstance(current_volume, pint.Quantity) and isinstance(total_mass, pint.Quantity): current_mass = current_volume * fuel_density mass_to_add = total_mass - current_mass return (mass_to_add / fuel_density).to_reduced_units() else: raise ValueError(\u0026#39;Provide the values as Pint physical quantities (pint.Quantity)\u0026#39;) Checking the units of the parameters to be passed:\nprint(current_vol) print(total_m) 7682 liter 49163.084467227694 pound Checking the result:\nrefuel_volume_implementation3(current_vol, total_m) 20088.85927770859 liter Great! Correct answer. Now the user can pass the parameters in any system of units, and pint will take care of converting internally.\nBut we are still not at an ideal implementation\u0026hellip;\nImagine a real program. In real code, this would be just one of the dozens of functions in the program. There would also be classes and other modules. In short, a whole universe of code to be maintained. Most likely, other parts of this code would also deal with physical quantities. Imagine checking in every function or method whether the correct type was passed to it? That would be a lot of repeated code with the same functionality, something that is best avoided.\nSince this is a package developed to deal with units and has a very active development community, surely someone has already thought of solving this kind of problem. For that, there is wraps.\nwraps is a decorator. The decorator concept provides a simple way to modify the behavior of a function without necessarily changing it. It is nothing more than a method to wrap a function, modifying its behavior.\nIn this case, wraps receives two parameters in the form of a tuple. The first refers to the units of the return value of the wrapped function and the second to the units of the parameters of that function. So, our new implementation becomes:\n@ureg.wraps((\u0026#39;l\u0026#39;), (\u0026#39;l\u0026#39;, \u0026#39;kg\u0026#39;, \u0026#39;kg/l\u0026#39;)) def refuel_volume(current_volume, total_mass, fuel_density=Q_(\u0026#39;0.803 kg/l\u0026#39;)): \u0026#39;\u0026#39;\u0026#39;Returns the volume of fuel that should refuel the airplane. Parameters ---------- current_volume: pint.Quantity, a value in liters is expected total_mass: pint.Quantity, a value in kilograms is expected fuel_density: pint.Quantity, by default \u0026#39;0.803 kg/l\u0026#39; Returns ------- volume to refuel: pint.Quantity, value in liters \u0026#39;\u0026#39;\u0026#39; current_mass = current_volume * fuel_density mass_to_add = total_mass - current_mass return mass_to_add / fuel_density Notice the changes made: removal of the isinstance check and removal of to_reduced_units. The first change is because wraps now takes care of the check. The second is because only values (magnitudes) are passed into the function. The units are checked beforehand and stripped, the calculations are done, and the units are applied to the return value. This improves the performance of the program.\nLet us check the behavior when values without units are passed:\nrefuel_volume(7682, 22300) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) \u0026lt;ipython-input-52-98280d6f7137\u0026gt; in \u0026lt;module\u0026gt; ----\u0026gt; 1 refuel_volume(7682, 22300) ~/Dropbox/work/cienciaprogramada/code/pint/.venv/lib/python3.8/site-packages/pint/registry_helpers.py in wrapper(*values, **kw) 263 # In principle, the values are used as is 264 # When then extract the magnitudes when needed. --\u0026gt; 265 new_values, values_by_name = converter(ureg, values, strict) 266 267 result = func(*new_values, **kw) ~/Dropbox/work/cienciaprogramada/code/pint/.venv/lib/python3.8/site-packages/pint/registry_helpers.py in _converter(ureg, values, strict) 147 ) 148 else: --\u0026gt; 149 raise ValueError( 150 \u0026quot;A wrapped function using strict=True requires \u0026quot; 151 \u0026quot;quantity or a string for all arguments with not None units. \u0026quot; ValueError: A wrapped function using strict=True requires quantity or a string for all arguments with not None units. (error found for l, 7682) Great, it raised an error, which is the behavior we want.\nLet us check the behavior when passing quantities:\ncurrent_vol = Q_(\u0026#39;7682 l\u0026#39;) total_m = Q_(\u0026#39;22300 kg\u0026#39;) refuel_volume(current_vol, total_m) 20088.85927770858 liter Cool, the expected result again. Finally, let us check what happens with this new version when a parameter is passed in another system of units.\ntotal_m.ito(\u0026#39;lb\u0026#39;) total_m 49163.084467227694 pound refuel_volume(current_vol, total_m) 20088.85927770858 liter Great!! Now we have our function implemented with best practices and safer to use.\nImproving the appearance of the result # Last but not least, let us see how to improve the look of our answer. In the context we are in, refueling an airplane with thousands of liters of fuel, surely any decimal place can be disregarded. After all, as we saw in the article about the airplane case, values are usually rounded. First, let us store the result in a variable:\nvolume = refuel_volume(current_vol, total_m) volume 20088.85927770858 liter The formatting options available for f-strings work with pint:\nprint(f\u0026#39;{volume:.2f}\u0026#39;) 20088.86 liter print(f\u0026#39;{volume:.0f}\u0026#39;) 20089 liter print(f\u0026#39;{volume:.0e}\u0026#39;) 2e+04 liter print(f\u0026#39;{volume:.2e}\u0026#39;) 2.01e+04 liter A more visual formatting, with unit symbols and powers of 10 (when they exist), can be obtained with ~P:\nprint(f\u0026#39;{volume:.2e~P}\u0026#39;) 2.01×10⁴ l In some situations, we may want to export the result for use in a document. And, in science and engineering, we use LaTeX a lot. So, we can express the result as LaTeX code:\nprint(f\u0026#39;{volume:.2e~L}\u0026#39;) 2.01\\times 10^{4}\\ \\mathrm{l} Users of the LaTeX siunitx package will be happy to know that it is possible to export using the package syntax:\nprint(f\u0026#39;{volume:.2e~Lx}\u0026#39;) \\SI[]{2.01e+04}{\\liter} If you have the babel package installed, you can even get translated units (here, into Brazilian Portuguese):\nvolume.format_babel(locale=\u0026#39;pt_BR\u0026#39;) '20088.85927770858 litros' # reusing the average_speed defined at the beginning of the article average_speed.format_babel(locale=\u0026#39;pt_BR\u0026#39;) '5.0 metros por segundos' Versions of the packages used in this document # This article was written in a Jupyter Notebook. The Jupyter Notebook is a graphical interface that allows editing notebooks in a web browser, such as Google Chrome or Firefox. Notebooks are virtual documents that allow code execution together with tools for text editing; that is, in addition to the usual programming routines, the user can document the whole process of producing the code. Exactly as was done here, with text explaining each piece of code. In this way, the notebook allows an interactive way of programming. It also allows more dynamic programming, offering the user the immediate output of the code; thus, there is no need to compile or run the whole document.\nI have already written here on the site about the Anaconda Project. When installing Anaconda, the Jupyter Notebook and JupyterLab programs are installed. Both allow you to interact with notebooks.\nIt is always important to know the versions of each package we use, because some features may change over time as each project\u0026rsquo;s development advances. The version_information package helps with this check in Jupyter Notebooks but unfortunately is not currently compatible with Python 3.8. Below I list the relevant version information used when writing this notebook:\nPython 3.8.5 IPython 7.18.1 pint 0.15 babel 2.8.0 notebook generated on Linux Mint 20 with Linux kernel 5.4.0 45 generic x86_64 with glibc2.29 The notebook is in this repository of mine on GitHub. The same repository also has the Pipfile files that allow you to create a virtual environment exactly like the one I created to write this article and the code shown. If you want to know more about virtual environments, how to create them from scratch and from files like Pipfile, read this article.\nConclusion # So, did you like it? It is a very powerful package and we have not even seen half of its features! But I hope you have realized that there are no more excuses for writing code that involves physical quantities without considering their units.\nIn the next article on the site, I will explore more aspects of using pint. After all, how do we use it with lists of values? Arrays? Combined with the package most used in science, numpy? And how do we use it when there are uncertainties associated with the values? And in plots? Find out here in this article\nSee you next time!\n","date":"2020-09-01","externalUrl":null,"permalink":"/2020/09/python-units-beer-pint/","section":"Blog","summary":"Whenever we work with measurements and experimental data, we need to express our values with their respective units. In this article, we will see a Python library for working with units and what that has to do with beer.","title":"Python, Units, and Beer(?!): The pint Package","type":"posts"},{"content":"","date":"2020-08-20","externalUrl":null,"permalink":"/tags/nasa/","section":"tags","summary":"","title":"NASA","type":"tags"},{"content":"Usually, one of the first topics we see when studying science is units and measurements. And then we run into some units that are not very common in many countries, including Brazil, such as feet and pounds. But we learn that there are conversion factors between these units and the ones we are used to, and this topic is usually considered simple and even boring.\nIn this article, we will see how NASA lost multimillion-dollar projects because of simple unit conversion errors. We will also see which system of units took humans to the Moon in the Apollo Program, by looking at the source code of the onboard computer.\nIf you prefer to watch a video, click the player below (the video is in Portuguese). The full article follows the video.\nMars Climate Orbiter (MCO) # On December 11, 1998, NASA launched the Mars Climate Orbiter (MCO), a 638 kg spacecraft whose goal was to study the Martian climate. However, on September 23, 1999, when the probe was maneuvering to enter orbit around Mars, communication with it was lost.\nArtist\u0026rsquo;s concept of the Mars Climate Orbiter. Source One of the reports produced afterward to understand the reasons for the accident determined that the main cause was the use of conflicting systems of units. The software used by the control team on Earth provided data in the United States customary system of units (similar to the imperial system), while the probe used units of the International System of Units (also known as SI, from the French Système International d\u0026rsquo;unités). As a result, incorrect instructions were sent to the probe, making it approach the planet too closely. It is believed that it was destroyed on entering the atmosphere or, if it managed to leave the atmosphere, was lost in space. Mission cost: $327.6 million.\nThe figure compares the planned trajectory with the actual one:\nComparison of the expected and actual trajectories of the Mars Climate Orbiter. Source According to the report, the ground system that calculated the probe\u0026rsquo;s impulse produced results in pound-force seconds (lbf s), while the trajectory calculation system expected the result in Newton seconds (N s). The conversion factor is approximately 4.45, as shown in the image below using the SpeedCrunch calculator.\nUnit conversion using the SpeedCrunch calculator NASA had hired the company Lockheed to build the mission\u0026rsquo;s software system, and the specifications required the whole system to use the International System of Units. However, NASA chose not to blame Lockheed for the accident, stating that NASA itself failed to perform the verifications and tests that would have warned of the discrepancy. The report points to a series of internal communication and procedural failures, even showing that part of the team had already detected something strange in the data during the probe\u0026rsquo;s journey, but decided to report it by e-mail(!) instead of opening a formal analysis procedure, which perhaps explains this stance by NASA. Let us look at some excerpts from the report:\nIn the MCO project, and again in the MPL project, there is evidence of inadequate communications between the project elements, including the development and operations teams, the operations navigation and operations teams, the project management and technical teams, and the project and technical line management. It was clear that the operations navigation team did not communicate their trajectory concerns effectively to the spacecraft operations team or project management. In addition, the spacecraft operations team did not understand the concerns of the operations navigation team.\nAn early comparison of these spacecraft-generated data with the tracking data might have uncovered the units problem that ultimately led to the loss of the spacecraft. When conflicts in the data were uncovered, the team relied on e-mail to solve problems, instead of formal problem resolution processes such as the Incident, Surprise, Anomaly (ISA) reporting procedure. Failing to adequately employ the problem tracking system contributed to this problem “slipping through the cracks.”\nThe report also shows that the software modules were tested separately, but due to file formatting failures (see page 17 of the report), the integration tests, where the output of one module is used as input to another, were postponed, and the team had little time for analysis.\nThe Software Interface Specification (SIS) was developed but not properly used in the small forces ground software development and testing. End-to-end testing to validate the small forces ground software performance and its applicability to the specification did not appear to be accomplished. It was not clear that the ground software independent verification and validation was accomplished for MCO. The interface control process and the verification of specific ground system interfaces was not completed or was completed with insufficient rigor.\nAs in most accidents, there were human components that were decisive in causing it. This excellent IEEE Spectrum article shows other aspects of this accident in more detail, with interviews. One section of the article shows that project participants reported decision-making problems that already existed at NASA earlier and resulted in the unfortunate Space Shuttle Challenger accident, which killed all 7 crew astronauts in 1986.\nThe Space Shuttle Challenger lifts off on its final mission. Source NASA\u0026rsquo;s decision-making and risk management process was also pointed to as one of the reasons for another space shuttle explosion four years after the loss of the MCO, in 2003, when Columbia disintegrated on reentering Earth\u0026rsquo;s atmosphere, also killing all 7 crew members.\nDemonstration for Autonomous Rendezvous (DART) # In 2005, only 6 years after the loss of the MCO at Mars, another NASA project ran into trouble. DART\u0026rsquo;s main goal was to test automatic navigation techniques, being able to dock automatically with other aerospace devices while avoiding collisions. The project had no means of interactive control from the ground, nor were there ways to update the software after launch. All operations in orbit depended solely on pre-programmed criteria. DART collided with the MUBLCOM satellite during an approach maneuver. Mission cost: 110 million dollars.\nIn the artist\u0026rsquo;s concept below, DART approaches the satellite:\nArtist\u0026rsquo;s concept of DART (bottom) approaching the MUBLCOM satellite (top). Source In a preview of the report produced by the accident analysis team, the software development process was harshly criticized. There were parameters in the code that were not used (leftovers), tests were not run for several scenarios, and there was a unit conversion error in one of the mathematical simulation models.\nIt is notable that a section of this preview states that the velocity measured by the instruments was \u0026ldquo;about\u0026rdquo; 0.6 meters per second more than it should have been (original: DART\u0026rsquo;s primary GPS receiver consistently produced a measured velocity that was offset or \u0026ldquo;biased\u0026rdquo; about 0.6 meters per second from what it should have been.). Some suspect that a unit conversion was done wrongly in the code. Since 1 ft/s is equivalent to 0.3048 m/s, the difference in magnitudes is 0.6952 or, using the scientific terms of the report (haha), about 0.6. Thus, it is suspected that some conversion factor from feet to meters, incorrectly done as an addition, might be in the code. The suspicion grows given that there is a mention of a unit conversion error in one of the mathematical simulation models (original: an omitted units conversion caused an error in a simulation math model.)\nUnfortunately, more details were not made public, since the full report was not released. According to NASA, there were details protected by the United States' ITAR, the International Traffic in Arms Regulations. But the report preview makes clear several problems in the software development workflow, just as in the MCO case. The report also clearly shows the concerns about communication and software development at NASA, which was also a target of the media at the time.\nNASA and the International System of Units # The trip to the Moon # NASA\u0026rsquo;s history with systems of units is quite long. For example, the Apollo Guidance Computer (AGC), the computer aboard the command module and the lunar module of the Apollo Project, stored data internally in the International System of Units, but displayed data and results in the United States customary system of units. This was because NASA\u0026rsquo;s operational-level personnel and the astronauts themselves understood the American system better.\nThe figure shows the computer and its keyboard:\nApollo Guidance Computer (AGC) and Display and Keyboard (DSKY). Source Thus, although internally altitude and velocity, for example, were stored in meters and meters/second, they were displayed in feet and feet/second. Likewise, masses were displayed in pounds, although stored in kilograms. The analysis of the source code that is available makes this very clear:\n# *** THE ORDER OF THE FOLLOWING SIX CONSTANTS MUST NOT BE CHANGED *** FDPS\t2DEC\t4.3670 B-7\t# 9817.5 LBS FORCE IN NEWTONS MDOTDPS\t2DEC\t0.1480 B-3\t# 32.62 LBS/SEC IN KGS/CS DTDECAY\t2DEC\t-38 FAPS\t2DEC\t1.5569 B-7\t# 3500 LBS FORCE IN NEWTONS MDOTAPS\t2DEC\t0.05135 B-3\t# 11.32 LBS/SEC IN KGS/CS ATDECAY\t2DEC\t-10 # ******************************************************************** FRCS4\t2DEC\t0.17792 B-7\t# 400 LBS FORCE IN NEWTONS FRCS2\t2DEC\t0.08896 B-7\t# 200 LBS FORCE IN NEWTONS SETLOC\tP40S1 BANK COUNT*\t$$/P40 # *** APS IMPULSE DATA FOR P42 *************************************** K1VAL\t2DEC\t124.55 B-23\t# 2800 LB-SEC K2VAL\t2DEC\t31.138 B-24\t# 700 LB-SEC K3VAL\t2DEC\t1.5569 B-10\t# FAPS (3500 LBS THRUST) # ******************************************************************** S40.136\t2DEC\t.4671 B-9\t# .4671 M NEWTONS (DPS) S40.136_\t2DEC\t.4671 B+1\t# S40.136 SHIFTED LEFT 10. In this snippet of code, we have specifications for the Lunar Module\u0026rsquo;s engine. DPS stands for Descent Propulsion System and APS for Ascent Propulsion System. Note that in the code comments, indicated by #, the respective values in the American system of measurement were placed.\nThe mission report itself is interesting from the point of view of systems of units. The more engineering-oriented part, with data on the spacecraft and the trip, uses the American system, while the part with the results of the experiments uses the international system. This reflects the fact that, even back then, the American scientific community used the international system more, probably because journals and other publications generally require this system of units.\nThe figure below shows an example of each part:\nFigures from the Apollo mission report. Left: lunar descent orbital events using the American system. Right: seismic experiment results using the International System of Units. The $300 million system-of-units migration # At some points in its history, NASA started movements toward the transition to the International System of Units. This link on NASA\u0026rsquo;s website is fascinating: it cites an American law that forces the use of SI in government-funded programs, unless its use is impractical. Governments and their brilliant legislators\u0026hellip; It is no surprise that, a few lines later, we find: \u0026ldquo;Although the use of SI in the US is increasing, the aerospace sector is recognized as an area where adoption will be difficult, due to the long-standing use of the US-based \u0026lsquo;inch-pound\u0026rsquo; system for aircraft.\u0026rdquo;. Well, OK then\u0026hellip;\nThe case of the Constellation Program is emblematic when it comes to systems of units. The program, started in 2005, was quite ambitious, proposing to complete the construction of the International Space Station and return to the lunar surface before 2020. We know that this did not happen: the program was canceled in 2010 because of its high costs, although some ongoing projects were redirected to other programs. The program had initially proposed that it would use the metric system, but\u0026hellip; internally a request was made that the metric system not be used, and a report was produced from the analysis of that request. Let us briefly analyze this report.\nAccording to the report, the Constellation Program team had estimated a cost of $368 million between 2009 and 2015 to make the transition to SI. A large part of this cost would come from reusing hardware and software from past programs, so that documentation and interface revisions would be needed. And the team would also have found it impractical to make the change because there were technical problems of other kinds that led to the need to cut costs and speed up procedures to meet deadlines.\nThe report thus estimated the cost of the conversion, year by year:\nFigure from the Constellation Program report: estimated cost of adopting the metric system, per fiscal year. The response given by the reviewers in the report is incredible. First, they criticized the fact that the program had raised the cost of adopting SI. OK, but what about the cost of NOT adopting it? A life-cycle cost analysis was not done. Considering the internationalization of space programs, not migrating to SI could cost more in the future, because a large part of the international community uses the metric system and training in the American system would be needed.\nAnother argument from the Program team is that the conversion would add unacceptable risks to the project. But the report questions why, then, in 2007, during the Program\u0026rsquo;s planning period, the metric system was selected on the team\u0026rsquo;s own justification that it was simpler, less error-prone, and aligned with NASA\u0026rsquo;s international policy. And it even cites the MCO case we have already described to show that changing policy involving systems of measurement with the project under way can lead to risks that are, in fact, unacceptable.\nThe Program team also tried to use the argument that parts and other components are generally manufactured with measurements compatible with the American system. However, the report states that metric measurements are equally available and that, again, the overall life cycle must be considered. With industries increasingly migrating to the metric system, it was not reasonable to start a new program in the American system. And, again, the text at this point reinforces the importance of considering international partnerships for parts, as well as commercial and personnel partnerships, all using the metric system.\nThe Program\u0026rsquo;s decision ended up conflicting with another US government agency, the National Institute of Standards and Technology (NIST). The report shows the communication between the two state entities, pointing out that NIST was not at all happy with NASA\u0026rsquo;s stance. This passage is notable (paraphrased): \u0026quot;[a] successful voluntary US transition to the metric system is a critical factor in the competitive success of companies and of the country. NASA is a flagship federal agency to which other agencies and industry look for leadership and inspiration.\u0026quot;\nFinally, the Program team\u0026rsquo;s last argument was that NASA\u0026rsquo;s use of the metric system was hindered by the fact that the Department of Defense does not use it in its projects. This would mean that aerospace companies would not have as many parts available for the metric system, given the great influence of Department of Defense contracts. Earlier in the report, it had already been shown that there are enough parts available and, at this point, the report even cites Department of Defense projects that were done entirely in SI.\nIt seems that the space cowboys do not mind picking fights much\u0026hellip; First NIST and now the Department of Defense. Someone there really does not like unit conversion factors and the metric system.\nThis decision was not well received by the private sector, as shown in this article from the time. Company representatives complained, rightly, that NASA\u0026rsquo;s delay in migrating to SI made it harder to create a global aerospace market, in addition to hindering international research collaborations. After all, a state-owned organization the size of NASA greatly impacted the market with every decision it made. Remember that we are talking about a little over a decade ago.\nCurrent situation # But how are we doing now, a decade after all of this happened? Has NASA succumbed to the International System of Units?\nWell, if you have read this article up to here, you must have realized that the system of units is the least of NASA\u0026rsquo;s problems. What becomes evident are the various communication and management issues of these projects. If a seemingly \u0026ldquo;simple\u0026rdquo; change of system of units costs millions of dollars, imagine the other factors involved in these projects. Each report clearly shows several project development problems, from the software to the equipment itself. The reality is that each project has become bigger and more expensive, and money, even government money, is finite.\nWith that in mind, since the mid-2000s, NASA has intensified collaboration programs with private companies such as Boeing, Orbital, Sierra Nevada, and the best known, SpaceX. These programs saved NASA millions of dollars. Obviously, there are still programs led by NASA, but a significant part of them are in cooperation with companies and space agencies from other countries such as Japan, Canada, and the European Space Agency. Thus, reality imposed itself, and when the American system of measurement is still used, it is on a smaller scale and internally. Equipment and software are (hopefully) developed in SI.\nThis does not mean that the culture of the people involved and of the institution has changed. A simple visit to NASA\u0026rsquo;s website shows this. Outreach materials still use the American system. For example, see this Space Launch System material, a launch rocket under development. The material is clearly aimed at Americans themselves, which is to be expected from an American government agency, isn\u0026rsquo;t it?\nNow look at this material on SpaceX\u0026rsquo;s Falcon vehicles. Throughout the document, the metric system is shown first, with the conversion to the American system in parentheses. After all, it is a service-providing company, so it will seek to communicate in a way that reaches as many customers as possible. Besides, the company\u0026rsquo;s marketing and public perception are also important.\nComparing NASA and SpaceX materials. Left: NASA with American units. Right: SpaceX with both systems, American and international If a large part of the world uses the metric system, it is essential for the company that people can understand its materials. This is notable in SpaceX\u0026rsquo;s launch videos:\nIn the videos, altitude is shown in kilometers and velocity in kilometers per hour. This drew press attention at the time of the first launches.\nTo get an idea of how much of our planet has already adopted the metric system, the following map shows the state of metrication around the world:\nCountries by current metrication status. Green: complete; yellow: nearly complete; orange: some adoption to partially complete; red: little adoption. Source Conclusion # So, have you ever stopped to think that the topic of conversion/systems of units could have so many ramifications? Although the systems and the conversions themselves are simple, there are several human factors involved, especially cultural ones. We also took advantage of the topic to have some discussions about software development. I hope you enjoyed the article, and don\u0026rsquo;t forget to leave your comments and share it.\nIf you want to read more about another situation where unit errors caused problems, see the case of the airplane that ran out of fuel in mid-flight.\nSee you next time.\n","date":"2020-08-20","externalUrl":null,"permalink":"/2020/08/nasa-loses-millions-unit-errors/","section":"Blog","summary":"In this article we will see how NASA has lost multimillion-dollar projects to simple unit conversion errors. We will also see which system of units took humans to the Moon in the Apollo Program, by looking at the source code of the onboard computer.","title":"NASA Loses Millions of Dollars to Unit Errors","type":"posts"},{"content":"","date":"2020-08-15","externalUrl":null,"permalink":"/tags/calculadora/","section":"tags","summary":"","title":"calculadora","type":"tags"},{"content":"","date":"2020-08-15","externalUrl":null,"permalink":"/tags/calculator/","section":"tags","summary":"","title":"calculator","type":"tags"},{"content":" How can a plane run out of fuel at 12 km of altitude? Today\u0026rsquo;s article is about a Boeing that became a 100-ton-plus \u0026ldquo;glider\u0026rdquo; and how it all happened because of a simple unit conversion error.\nIf you prefer to watch a video (in Portuguese), click the player below. The full article follows the video.\nAbout the plane # Air Canada Boeing 767-200 that operated flight 143 In 1978 Boeing began developing the 767 line, its first twin-engine widebody (twin-aisle) aircraft, which came with several technological novelties, especially electronic displays. In the money of the time, the aircraft cost 40 million dollars. The plane in our story is the Boeing 767-200 that operated Air Canada flight 143 between Montreal and Edmonton, with a stop in Ottawa, on July 23, 1983. The flight crew was very experienced, with Captain Robert Pearson having more than 15,000 flight hours and First Officer Maurice Quintal more than 7,000 flight hours.\nIn the photo, Captain Pearson:\nCaptain Robert Pearson The incident # While the plane was flying over Red Lake, Ontario, at an altitude of 41,000 feet (12.5 km), the cockpit warning system alarm went off, indicating a fuel pressure problem. The pilots assumed it was a fuel pump problem and turned the pump off, since gravity would still deliver fuel to the engines. The onboard computer indicated (wrongly, as we will see later) that there was enough fuel to fly. A short time later the left engine flamed out and the pilots, in contact with air traffic control, diverted the flight to Winnipeg. Air traffic control said they were 120 nautical miles (222 km) from Winnipeg airport.\nWhile they were changing course and talking to air traffic control, a new alert came: this time indicating that all the engines had failed and, shortly after, the right engine stopped working. The 767 was one of the first planes to have an Electronic Flight Instrument System, which ran on electricity generated by the engines. With no engines, the system became inoperative, leaving only a few instruments powered by the emergency battery. The remaining instruments gave enough information to land the plane. However, there was no information about the aircraft\u0026rsquo;s vertical speed, which would indicate its rate of descent, a piece of information needed to calculate how long the aircraft could keep gliding. At that moment, the Winnipeg control tower said they were about 65 nautical miles from Winnipeg.\nThe engines also powered the plane\u0026rsquo;s hydraulic systems. With their loss, a backup system is activated, which consists of turbines that generate power from the movement of the air. So, as the speed decreased, less power was available to the hydraulic systems, which are essential for good control at landing.\nThe glider Boeing # Situations like this were not part of the pilots\u0026rsquo; training, nor did the aircraft\u0026rsquo;s emergency instructions describe the procedures to follow in a no-engine situation. Luckily, Captain Pearson had a lot of experience with gliders and used all that knowledge to avoid a disaster.\nThe first piece of information Pearson needed was the speed he had to maintain to glide. From practical knowledge, he estimated it would be 220 knots (407 km/h). As written earlier, the plane did not display the rate of descent. So he and Quintal stayed in constant contact with air traffic control, since control could report the altitude from the information collected by radar. They then estimated that for every 10 nautical miles the plane lost 5,000 feet of altitude. With that he could calculate the glide ratio, that is, the ratio between the horizontal distance traveled and the height lost. Let\u0026rsquo;s do this calculation, taking the opportunity to convert the data to the International System of Units:\n$$10\\ \\text{M} \\approx 18.52\\ \\text{km}$$$$5{,}000\\ \\text{ft} \\approx 1.524\\ \\text{km}$$$$\\begin{aligned} \\text{glide ratio} \u0026= \\frac{18.52\\ \\text{km}}{1.524\\ \\text{km}} \\\\ \u0026= 12.15 \\approx 12:1 \\end{aligned}$$Here, M is the symbol for nautical mile and ft is the symbol for feet.\nThat is, for every 12 km they advanced horizontally, they lost 1 km of height. Just for comparison, gliders usually have glide ratios above 30:1. Notice, in the following figure, that the glide ratio is nothing more than the cotangent of the descent angle:\nThe glide ratio is the cotangent of the descent angle. Note the proportion of the axes. At the moment they were making these calculations, they were at 35,000 feet of altitude, so the maximum they could advance is:\n$$\\begin{aligned} \u002612 \\times 35{,}000\\ \\text{ft} \\\\ \u0026\\quad \\times 3.048 \\times 10^{-4}\\ \\text{km}\\ \\text{ft}^{-1} \\\\ \u0026\\approx 128\\ \\text{km} \\end{aligned}$$As written earlier, at the moment they were making these calculations they were about 65 nautical miles from Winnipeg, which is equivalent to 120 km. Mathematically, then, it seems they could reach Winnipeg. But, since the plane\u0026rsquo;s speed would have to decrease on the approach for descent, it would not be possible to go to Winnipeg. In addition, since the plane\u0026rsquo;s main system was disabled, the position given by radar was not very accurate because the plane\u0026rsquo;s transponder was not working, so they needed a margin of error in these values.\nQuintal then proposed landing at an airfield in Gimli, where he had served as a pilot in the Canadian Air Force. The distance to Gimli was 45 nautical miles (83 km), which would be safer. Let\u0026rsquo;s see on the map roughly what the situation was in terms of distances.\nMap with the positions of the main moments of the incident. At the moment of the loss of the second engine, the plane was 83 km from Gimli and 120 km from Winnipeg. Without power, the landing gear had to be lowered by gravity. However, the nose landing gear did not lock in the correct position. To reduce altitude and speed for landing, Pearson banked the aircraft to expose more area, increasing drag and decreasing the flight height. And as they approached the runway it also became evident that there was some activity going on. And there really was. Part of the runway was being used for a go-kart race and there were family campsites around the track! And, since the plane had no engines, it made no noise, so the public only realized what was happening when the plane was almost touching the runway.\nAir Canada flight 143 plane after the emergency landing on a go-kart track, near a campsite When the main landing gear touched the runway, Pearson braked with all his strength, blowing out two tires. Since the aircraft\u0026rsquo;s nose landing gear had not locked, the nose started to drag on the ground. Fortunately this helped reduce the speed and the plane did not leave the runway, nor did it hit any of the people or campsites. None of the 61 passengers were seriously injured; some had minor injuries during the evacuation because the rear of the plane was very high and the slides were not tall enough.\nThe photo shows the position where the plane stopped:\nSlides that did not properly reach the ground because of the plane\u0026rsquo;s final position But after all, why did all this happen?\nThe unit mix-up # The accident report shows, as always in the case of large-scale accidents, that there was a series of factors that culminated in the forced landing at Gimli. Several internal communication problems were pointed out, in addition to failures to follow standard procedures. But one aspect is of interest to us here: a unit conversion failure.\nAt the time of the incident, the Canadian aviation sector was in the process of migrating to the metric system (the International System of Units). The Boeing 767 was the first plane acquired by Air Canada whose instruments were already in the metric system (liters and kilograms), instead of imperial units (gallons and pounds). All the company\u0026rsquo;s other aircraft used imperial units.\nFor the trip to Edmonton, the pilot calculated that 22,300 kg (49,163 lb) of fuel would be needed. The company\u0026rsquo;s maintenance technicians checked that there were already 7,682 liters in the tank. To calculate how much fuel should be added, the crew had to convert the volume to mass, subtract it from the total mass required, and convert back to volume.\nThe density to be used was 0.803 kg/l. So, the correct calculations would be:\nFuel mass already present: \\(7{,}682 \\text{ L} \\times 0.803 \\text{ kg L}^{-1} = 6{,}169 \\text{ kg}\\) Fuel mass to add: \\(22{,}300 \\text{ kg} - 6{,}169 \\text{ kg} = 16{,}131 \\text{ kg}\\) Conversion to volume: \\(\\dfrac{16{,}131 \\text{ kg}}{0.803 \\text{ kg L}^{-1}} = 20{,}088 \\text{ L}\\) However, since the custom was to use the imperial system of units, the staff understood that 22,300 pounds of fuel were needed and used a density value of 1.77 lb/l. The volume of fuel, like that of any liquid, varies with temperature. So, it is normal for the maintenance crew to consult density versus temperature tables to know which value to use. In this case, the fuel supplier\u0026rsquo;s instructions indicated that this was indeed the density for that day\u0026rsquo;s conditions. Since the custom was to work in pounds and use values close to this one, no one noticed the error. So, for them the calculations were:\nFuel mass already present: \\(7{,}682 \\text{ L} \\times 1.77 \\text{ lb L}^{-1} = 13{,}597 \\text{ lb}\\) Fuel mass to add: \\(22{,}300 \\text{ lb} - 13{,}597 \\text{ lb} = 8{,}703 \\text{ lb}\\) Conversion to volume: \\(\\dfrac{8{,}703 \\text{ lb}}{1.77 \\text{ lb L}^{-1}} = 4{,}917 \\text{ L}\\) Thus, instead of fueling with 20,088 liters, the plane was fueled with only 4,917 liters (in fact, 5,000 liters, by rounding according to the report).\nFar beyond a simple unit error\u0026hellip; # The accident report shows that it was a collective error. Neither the maintenance crew nor the pilots, after the causes of the accident were investigated, could answer exactly what the conversion procedure was. The calculations, however simple, were done by many hands and no one noticed that the density was in the wrong unit. All the checks that were made were purely numerical, since no one suspected the units.\nThe report also points out that, had those involved consulted the flight operations manual, the conversion procedure would have been described correctly. However, those involved were not in the habit of following this procedure because, on other company flights with older planes, the crew included a flight engineer, who was responsible for these calculations. On this modern plane, the company considered a flight engineer unnecessary but did not reassign this checking duty to anyone else.\nAnother link in this incident is the fact that the fuel indication system was inoperative. On this plane, it was a dual system, each processor channel calculating the amount of fuel independently. So, even if one failed, there was still the other. And, in this kind of situation, the procedure is to check the amount of fuel manually at each stop, with the aid of rulers, by maintenance technicians. The check was done both in Montreal and in Ottawa, but on this flight both channels were faulty. Thus, the aircraft should not have been cleared to fly according to the procedures described in the flight manuals.\nHowever, due to several misunderstandings, Pearson understood that he could fly if the amount of fuel had been checked by the maintenance technicians. What contributed to this understanding was that the procedures in this aircraft\u0026rsquo;s manuals, more specifically in the Minimum Equipment List (MEL), changed often, because the plane was still new and adjustments were being made, so it was common for flights to be cleared by maintenance technicians even when contradicting the manual\u0026rsquo;s guidelines.\nWith the indication system inoperative, the pilots had to manually enter the amount in the fuel tanks into the onboard computer so that it could calculate fuel consumption. So, the value entered was wrong.\nConclusion # Although it seems that the unit conversion was the main cause of this accident, we see that, if all the procedures had been followed correctly, there would have been several chances for the error to be noticed. Several layers of safety were broken, by several characters in the story.\nMoreover, although the pilots\u0026rsquo; skill was essential to save the lives of everyone on board, their errors, uncovered by the investigation, of not following procedures led to their being removed from the company\u0026rsquo;s activities for a while. Air Canada was also penalized for having operations that allowed this whole chain of poor communication and that allow activities to be carried out without being provided for.\nHow can you avoid making this kind of mistake in your professional life? First, follow formally established procedures. After all, that is what they are for. And, if you are going to do calculations, write the values with their respective units. By the way, I have already written here about a calculator that makes calculations with units easier, so be sure to check it out. Dimensional analysis is simple and would easily reveal conflicting units.\nSee you in the next article!\n","date":"2020-08-15","externalUrl":null,"permalink":"/2020/08/unit-conversion-error-plane/","section":"Blog","summary":"How can a plane run out of fuel at 12 km of altitude? Today’s article is about a Boeing that became a 100-ton-plus “glider” and how it all happened because of a simple unit conversion error.","title":"Can a Unit Conversion Error Bring Down a Plane?","type":"posts"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/ambiente-virtual/","section":"tags","summary":"","title":"ambiente virtual","type":"tags"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/conda/","section":"tags","summary":"","title":"conda","type":"tags"},{"content":"In this article we will see how to use virtual environments to manage Python projects. We will see that it is possible to have a different Python version for each project, and also how packages and dependencies are controlled.\nIf you prefer to watch a video, click the player below (the video is in Portuguese). The full article follows the video.\nWhy use virtual environments? # Like other programming languages, Python has its own way of downloading, storing, and managing packages and modules. Of particular interest to us is understanding how and where these packages are stored.\nMost Unix-derived systems, such as macOS and the most common Linux distributions, already come with some Python version preinstalled, since some system tools depend on the language. I am a Linux user and currently have Linux Mint 20 installed, so the examples here come from that distribution, but they can easily be replicated on others and on a Mac. Windows has no system tools that depend on Python, so it usually does not come with any version preinstalled, but installing one is very easy, as a simple internet search will show.\nAs described above, Linux distributions have tools that depend on Python. Even though version 3 of the language was released in 2008, many projects still use some micro version of Python 2, often for backward-compatibility reasons. The last micro version of Python 2 was 2.7.18, released in April 2020, and its support ended in early 2020. There are several interesting discussions about this long support period for such an old version; I recommend starting here if you are interested in the subject. And if you are not quite sure how version numbers are represented in Python, read here.\nIt is easy to tell whether both versions are present on your Linux system. In the terminal, type which python and then which python3. The paths returned are the locations of the Python 2 and Python 3 interpreters, respectively.\n$ which python3 /usr/bin/python3 In distributions based on Ubuntu 20.04, such as Linux Mint 20, which I use, Python 2 was officially removed from the repositories, but it can still be accessed with the python2 command (and not just python, as was customary in earlier distributions).\nBy default, the shell that opens when you type python in the terminal in most distributions is Python 2. To open a Python 3 shell, just use the python3 command.\nMost system packages are stored in folders inside a system folder. To identify that folder, we can open a Python shell in the terminal and enter the following commands:\n\u0026gt;\u0026gt;\u0026gt; import sys \u0026gt;\u0026gt;\u0026gt; sys.prefix The result will usually be the system folder /usr. That folder has the following structure, with folders for executables, libraries, and source code.\n# tree /usr -L /usr ├── bin ├── games ├── include ├── lib ├── libexec ├── local ├── sbin ├── share └── src You can read the documentation of sys.prefix here.\nThird-party packages, usually installed with the pip manager, will be installed in one of the folders given by the commands:\n\u0026gt;\u0026gt;\u0026gt; import site \u0026gt;\u0026gt;\u0026gt; site.getsitepackages() [\u0026#39;/usr/local/lib/python3.8/dist-packages\u0026#39;, \u0026#39;/usr/lib/python3/dist-packages\u0026#39;, \u0026#39;/usr/lib/python3.8/dist-packages\u0026#39;] Documentation of the site package.\nNotice that these are system folders: all projects will use these same folders. Let us consider the following scenario to see why this is a problem.\nSuppose you have two projects: Project A and Project B. Both depend on the Pandas library, but Project A is older and was built on version 0.25.3, while Project B is brand new and uses version 1.0.5, which is not fully backward compatible with 0.25.3. So Project A cannot be upgraded without rewriting some pieces of code, which is not feasible if the project is to be delivered on time. As the system stands, a single version of Pandas will be installed for the whole system, in a pandas folder inside the folder returned by the commands above, making it impossible to work on both projects at the same time.\nThis is where virtual environments come into play.\nWhat is a virtual environment? # In essence, virtual environments let you create an isolated environment for each Python project. That way, each project can have its own dependencies, regardless of the dependencies of other projects.\nIn our example, we just need to create one environment for Project A, where Pandas 0.25.3 will be installed, and another for Project B, where version 1.0.5 will be installed.\nThere is no limit to how many environments you can create, other than the physical space on your disk, since they are just directories. Let us look at the main tools available for creating and managing virtual environments.\nvenv # The first tool we will see for creating virtual environments is venv. It is part of the Python 3 standard library, so there is nothing to install. In this article we will focus only on Python 3, given that support for Python 2 has been discontinued.\nFirst, we need a folder for each project:\nmkdir projects \u0026amp;\u0026amp; mkdir projects/project_{A,B} Structure of a venv environment # Now, inside each folder, we will create a virtual environment which, at first, we will think of as a simple folder inside the project. It is common to name this folder env or .env (remember that a leading dot marks a hidden folder). You can use any name, but we will keep to the convention here and use .env. So, inside each project folder, we run the command\npython3 -m venv .env Notice that the command makes it explicit that the interpreter of these environments will be the Python 3 interpreter. Let us look at the structure of the .env directories created in each project:\n$ tree .env -L 3 .env ├── bin ├── activate ├── easy_install ├── pip ├── python -\u0026gt; python3 ├── python3 -\u0026gt; /usr/bin/python3 ├── include ├── lib ├── python3.8 ├── site-packages ├── lib64 -\u0026gt; lib ├── pyvenv.cfg └── share └── python-wheels Among these folders, we can highlight:\nbin: files that interact with the virtual environment; include: a copy of the Python version along with the site-packages folder where each dependency will be installed. There are also copies and symbolic links (symlinks) for some tools and the interpreter executables. All of this ensures that Python commands and code run within the context of the current environment, guaranteeing isolation. We will see more details soon.\nAlso in the bin folder there are activate scripts which, as the name suggests, are used to actually activate the environment so that the shell uses the environment\u0026rsquo;s Python executable and site-packages folder, not the system\u0026rsquo;s.\nActivating a venv environment # To activate the environment, inside the project folder, run\n$ source .env/bin/activate (.env) $ Notice that the prompt now shows the environment name, .env, as a prefix, indicating that it is active.\nUnderstanding how venv works # Now, when we run which python, the path that appears refers to the virtual environment and no longer to the system (compare with the output at the beginning of the article). So, in the folder of our Project A:\n$ which python /home/chico/projects/project_A/.env/bin/python Let us now deactivate the virtual environment. All it takes is the following command\n$ deactivate With the environment deactivated, let us look at the $PATH environment variable. As a reminder, $PATH tells the shell which directories to search for executables in response to user commands. Let us activate the environment again to see how $PATH changed:\n$ echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games (.env) $ echo $PATH /home/chico/projects/project_A/.env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games Notice that, with the environment activated, the environment\u0026rsquo;s bin directory is the first path in $PATH. In practice, this means it will be the first directory the system searches for executables, so the shell will use the environment\u0026rsquo;s Python instance and not the one available to the whole system.\nSolving our scenario # In our scenario, we have two projects that depend on the Pandas package, but on different versions of it. In the Project A folder, after activating the environment, we need to run the following command to install the specific version of the package in question:\n(.env) $ pip install pandas==0.25.3 The command will install the Pandas package in the requested version and the necessary dependencies, such as the NumPy package. After the installation, we can check the packages in the environment with the command\n(.env) $ pip list Package Version ----------------- ---------- numpy 1.19.1 pandas 0.25.3 pip 20.0.2 pkg-resources 0.0.0 python-dateutil 2.8.1 pytz 2020.1 setuptools 44.0.0 six 1.15.0 Let us now deactivate the Project A environment and go to the Project B directory, activating the environment of this latter project. Since this project has just started, we want to begin with the latest Pandas version, so there is no need to specify the version: the latest one will be installed, which at the time of writing is 1.1.0.\n(.env) $ pip install pandas Running pip list again, we see that the latest version was indeed installed. So the two projects are isolated and have the versions they need.\n(.env) $ pip list Package Version ----------------- ---------- numpy 1.19.1 pandas 1.1.0 pip 20.0.2 pkg-resources 0.0.0 python-dateutil 2.8.1 pytz 2020.1 setuptools 44.0.0 six 1.15.0 Sharing a venv environment # In an even more realistic scenario, the projects would have many more dependencies, and installing them one by one, especially when a project is already structured as in the case of Project A, is unproductive. So, in projects that use venv, it is quite common to have a requirements.txt file. To generate the file from an already structured environment:\n(.env) $ pip freeze \u0026gt; requirements.txt In our simplified case of Project A, this is the structure of the file:\n(.env) $ cat requirements.txt numpy==1.19.1 pandas==0.25.3 python-dateutil==2.8.1 pytz==2020.1 six==1.15.0 It is a very simple structure, showing the project\u0026rsquo;s dependencies and their versions.\nTo create an environment from the file, you create a folder for the project with an environment as per the instructions above, copy the requirements.txt file to the folder, activate the environment, and then run the command:\n(.env) $ pip install -r requirements.txt That way, all collaborators on the project can have the same development environment, quickly and easily.\nMaking the environment show up in Jupyter Notebook # If the project uses Jupyter Notebook, for the environment\u0026rsquo;s interpreter and packages to be loaded you need to install and run, inside the environment:\n(.env) $ pip install ipykernel (.env) $ ipython kernel install --user --name=project_name Removing the environment # Since an environment created with venv is built around the virtual environment folder, to delete the environment you just deactivate it and remove the folder:\n(.env) $ deactivate $ rm -r .env conda # Many people start with Python through distributions such as Anaconda, which aim to simplify the management and development of Python projects, especially those aimed at the scientific field. Those who start this way end up coming into contact with the conda manager, which, by the way, works for any language, not just Python. conda can also manage virtual environments. These last sentences already highlight important differences between conda and pip, which many beginners wrongly consider equivalent, but there are others that go beyond the scope of this article, so I strongly recommend this article on the Anaconda project website. It is also worth noting that the approach and structure of conda virtual environments differ significantly from those of venv, so I advise reading the documentation of conda, which presents these differences in detail and describes the contexts where one approach or the other is more suitable.\nWe have already covered installing Anaconda here on the site. Check it out at this link. Since the focus of this article is creating and managing virtual environments, let us get straight to the point.\nBy default, environments created by conda are installed in the envs folder inside the conda directory. This way, the commands below can be run in any directory, since they do not affect it. You can change this behavior to have the environment inside a folder in your project, similar to venv, but there are some considerations about this change that go beyond the scope of this article, so I recommend reading the documentation on the subject.\nUnderstanding the base environment # When the Anaconda installation finishes, a virtual environment containing the Python interpreter and the installation\u0026rsquo;s packages, named base, is created automatically. The default installation settings make the environment activate automatically, so when you open the terminal it should show the environment in the prompt prefix. We can check which Python interpreter will be used:\n(base) $ which python /opt/anaconda/bin/python As expected, the path shown is the Anaconda installation directory.\nIf you want to list all the packages installed by Anaconda:\n(base) $ conda list The result should be a long list of packages.\nCreating and activating a conda environment # Considering our two-project scenario, let us create an environment for Project A. To create an environment:\n(base) $ conda create --name env_project_A An environment created this way will be completely empty, with no packages and no interpreter. Once you have an environment created, to activate it all you need is:\n$ conda activate env_project_A With the environment activated, we can verify that it is indeed empty:\n(env_project_A) $ which python (env_project_A) $ conda list Neither command returns anything.\nInstalling a Python interpreter in a conda environment # To install a Python interpreter in the environment you created:\n(env_project_A) $ conda install python Installing this way will install the latest Python version available in the conda repositories. If you want a specific version, just specify it:\n(env_project_A) $ conda install python=3.7 With an interpreter installed, we can check its path:\n(env_project_A) $ which python /home/chico/.conda/envs/env_project_A/bin/python Installing dependencies in a conda environment # To install packages in the environment, just pass the environment name and the package name. If only the name is passed, the latest version of the package available in the conda repository will be installed. If you want a specific version, just pass the version number. Take a look:\n(env_project_A) $ conda install -n env_project_A scipy (env_project_A) $ conda install -n env_project_A pandas=0.25.3 Creating a conda environment with interpreter and dependencies # Everything we have done so far could be done in a single command when creating the environment. That is, you can create an environment and already pass it the Python version to be used and the packages you want:\n$ conda create --name env_project_A python=3.7 scipy pandas=0.25.3 Use the conda list command to check the installed packages and their respective versions.\nHere we already see a big advantage of conda: the ability to install environments with different Python versions. Another advantage is that conda automatically checks for dependency conflicts among the packages to be installed, which is very useful in large projects with many dependencies, especially when updating them.\nChecking existing conda environments # To check which environments managed by conda exist:\n(base) $ conda info --envs # conda environments # env_project_A /home/chico/.conda/envs/env_project_A env_project_B /home/chico/.conda/envs/env_project_B base * /opt/anaconda The environment marked with * is the one currently active.\nSharing a conda environment # After setting up the environment with the packages you are interested in, it is worth exporting these settings so that others involved in the project can have an identical development environment, or so that you can replicate the environment at some point in the future. conda has a logic similar to venv and its requirements.txt file, except that in the case of conda the format is YAML and the file structure is a bit different to conform to the way conda works. To generate this file the environment must be active and, usually, the file is named environment.yml, although it can have any name:\n(env_project_A) $ conda env export \u0026gt; environment.yml Let us look at the structure of this file:\nname: env_project_A channels: - defaults dependencies: - _libgcc_mutex=0.1=main - blas=1.0=mkl - ca-certificates=2020.6.24=0 - certifi=2020.6.20=py37_0 - intel-openmp=2020.1=217 - ld_impl_linux-64=2.33.1=h53a641e_7 - libedit=3.1.20191231=h14c3975_1 - libffi=3.3=he6710b0_2 - libgcc-ng=9.1.0=hdf63c60_0 - libstdcxx-ng=9.1.0=hdf63c60_0 - mkl=2020.1=217 - mkl-service=2.3.0=py37he904b0f_0 - mkl_fft=1.1.0=py37h23d657b_0 - mkl_random=1.1.1=py37h0573a6f_0 - ncurses=6.2=he6710b0_1 - numpy=1.19.1=py37hbc911f0_0 - numpy-base=1.19.1=py37hfa32c7d_0 - openssl=1.1.1g=h7b6447c_0 - pandas=0.25.3=py37he6710b0_0 - pip=20.2.1=py37_0 - python=3.7.7=hcff3b4d_5 - python-dateutil=2.8.1=py_0 - pytz=2020.1=py_0 - readline=8.0=h7b6447c_0 - setuptools=49.2.1=py37_0 - six=1.15.0=py_0 - sqlite=3.32.3=h62c20be_0 - tk=8.6.10=hbc83047_0 - wheel=0.34.2=py37_0 - xz=5.2.5=h7b6447c_0 - zlib=1.2.11=h7b6447c_3 prefix: /home/chico/.conda/envs/env_project_A It is a somewhat more complex structure than the requirements.txt we saw earlier with venv. It shows the environment name, the channel to download packages from, the Python interpreter version for the environment, and the path the file was exported from.\nTo create an environment from a previously created environment file:\n$ conda env create -f environment.yml Deactivating and removing a conda environment # To deactivate the environment in use:\n(env_project_A) $ conda deactivate To completely delete an environment and the packages installed in it:\nconda remove --name your_env_name --all pyenv # Even though we have seen that the operating system already comes with Python, the available version is often not the latest. And it is never a good idea to directly upgrade the operating system\u0026rsquo;s version, because some applications may break, especially if they depend on a language feature that was removed in a later update with no backward-compatibility option. We have already seen that conda lets you install different Python versions per environment, but it is not the only option. Now we will talk about pyenv.\nThe project repository is very instructive, even explaining some things already covered in this article, such as how $PATH works.\nInstalling pyenv # It is advisable to check the repository for the installation instructions, because the project needs some dependencies to work. I will list below the ones required at the time of writing, but I recommend checking the repository to make sure nothing has changed:\n$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \\ libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \\ xz-utils tk-dev libffi-dev liblzma-dev python-openssl git $ curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash At the end of the previous command, instructions appear on the screen to be added to the configuration file of bash (or of any other shell you use, such as ZSH). So, at the end of your .bashrc file, or equivalent, add what appears on the screen, which should be something similar to the following:\nexport PATH=\u0026#34;~/.pyenv/bin:$PATH\u0026#34; eval \u0026#34;$(pyenv init -)\u0026#34; eval \u0026#34;$(pyenv virtualenv-init -)\u0026#34; Restart your shell for the changes to take effect:\n$ exec $SHELL Checking whether it was installed:\n$ pyenv If all goes well, the usage instructions and the main available commands will appear on the screen.\nUsing pyenv # To check all the interpreter versions available for installation:\n$ pyenv install -l Notice that there are many versions, including some still under development (with the -dev suffix).\nTo install a specific version:\n$ pyenv install 3.8.5 You can check the versions available on the system:\n$ pyenv versions * system 3.8.5 Notice that it highlights which version is the system\u0026rsquo;s original (system). The version marked with an asterisk is the one currently active.\nTo avoid the risk of modifying the system version, you can set the default Python of your choice:\n$ pyenv global 3.8.5 You may need to restart the terminal to see the effect of the command above. Confirming which Python is active:\n$ which python /home/chico/.pyenv/shims/python $ python -V Python 3.8.5 It is also worth checking that pip is using the same path shown by the which python command:\n$ pip --version That way, we make sure the installed packages are really tied to the Python we want and not to the system Python.\nNow you can combine this pyenv feature, which gives you several Python versions if you need them, with venv to have environments with those versions.\npipenv # Last but not least, we have pipenv. As you can see in the project repository, it is a very ambitious tool that aims to:\nunify the behavior of pip and venv; fix the problems that can occur in projects with requirements.txt; increase security by using hashes (cryptographic signatures); detail the project\u0026rsquo;s dependencies clearly; separate development dependencies from runtime dependencies. Installing pipenv # Installation is quite simple:\n$ pip install pipenv To confirm that the installation was done correctly, just use the pipenv command. A list of the main commands will be shown.\nKeeping the pipenv environment folder in the project folder # Like conda, pipenv by default sets up all environments in a system folder. Some people may prefer each environment to stay in the folder of its project. To do that, add the following line to ~/.bashrc (or the equivalent for your shell):\nexport PIPENV_VENV_IN_PROJECT=1 Restart the shell or run source ~/.bashrc for the change to take effect. Throughout the rest of this article, I will assume this was done and, therefore, that the environment folder is inside the project folder.\nCreating a virtual environment with pipenv # To use it, enter the project folder. Now, the following command will already create a virtual environment for the project:\n$ pipenv install An environment will be created with the Python interpreter active at the time of creation. We can verify that the environment was created by looking at the new items in the project folder:\n$ ls -l -a Pipfile Pipfile.lock .venv Structure of a Pipfile # There is a .venv folder, whose structure is similar to the one we described when talking about venv, and two Pipfile files. We can think of these files as similar to the requirements.txt or environment.yml files mentioned in other sections. However, the way pipenv manages dependencies is more robust.\nFirst let us investigate the Pipfile file:\n$ cat Pipfile [[source]] name = \u0026#34;pypi\u0026#34; url = \u0026#34;https://pypi.org/simple\u0026#34; verify_ssl = true [dev-packages] [packages] [requires] python_version=\u0026#34;3.8\u0026#34; We see that it has the repository source, PyPI; the parameter that says to use SSL; and the Python version. Notice that there are sections, still blank because this is a new project, that separate the development packages from the project\u0026rsquo;s actual packages.\nInstalling a dependency with pipenv # To install a dependency, for which we will use Pandas in our Project A as an example, we run:\npipenv install pandas==0.25.3 If no specific version were used, the latest version would be installed.\nLet us look at the Pipfile again:\n$ cat Pipfile [[source]] name = \u0026#34;pypi\u0026#34; url = \u0026#34;https://pypi.org/simple\u0026#34; verify_ssl = true [dev-packages] [packages] pandas = \u0026#34;==0.25.3\u0026#34; [requires] python_version=\u0026#34;3.8\u0026#34; We see that it lists the Pandas package with the requested version. If no version had been passed, a \u0026ldquo;*\u0026rdquo; would be shown next to the package name.\nWe know that Pandas has its own dependencies and that these were certainly installed too. To see how these dependencies relate to each other and to Pandas:\n$ pipenv graph pandas==0.25.3 - numpy [required: \u0026gt;=1.13.3, installed: 1.19.1] - python-dateutil [required: \u0026gt;=2.6.1, installed: 2.8.1] - six [required: \u0026gt;=1.5, installed: 1.15.0] - pytz [required: \u0026gt;=2017.2, installed: 2020.1] Notice that we now see all the Pandas dependencies and which version ranges are compatible. This enables the so-called deterministic build, where you know exactly all the versions used in the project. This management is very important when you develop locally and plan to deploy to production exactly what was developed locally. For this fine-grained control, we have the Pipfile.lock file, a long file describing all direct and indirect dependencies, with their cryptographic signatures, sources, and versions.\nInstalling development dependencies # Over the course of a project, we will have some dependencies that are useful only during development and do not need to go to production. Examples are testing libraries such as pytest, or libraries we use to keep the code style in line with the language standards, such as flake8. To install a dependency for development only:\n$ pipenv install --dev pytest $ pipenv install --dev flake8 Such dependencies are in the [dev-packages] section of the Pipfile.\n$ cat Pipfile [[source]] name = \u0026#34;pypi\u0026#34; url = \u0026#34;https://pypi.org/simple\u0026#34; verify_ssl = true [dev-packages] pytest = \u0026#34;*\u0026#34; flake8 = \u0026#34;*\u0026#34; [packages] pandas = \u0026#34;==0.25.3\u0026#34; [requires] python_version=\u0026#34;3.8\u0026#34; Sharing a pipenv environment # To start a project from a Pipfile, as when, for example, we clone a project from GitHub, we run:\n$ pipenv install -d Notice that the -d flag was passed, indicating that development dependencies will also be installed. If that is not necessary, just drop the flag.\nWhen we run the previous command, the whole dependency graph will be rebuilt. If you want a deterministic build with the exact versions present in the Pipfile.lock file, we use the sync command, which has the -d flag to also install development dependencies:\n$ pipenv sync -d Activating a pipenv environment # pipenv has its own shell that can be activated:\n$ pipenv shell (project A) $ Now we can use the installed libraries\n(project A) $ pytest . To deactivate, just use the exit command.\nIf you want to run a command without having to enter and leave the shell, we can use run:\n$ pipenv run pytest . Conclusion # In this article we saw how Python dependencies are stored and managed by different tools, as well as how we can have different Python versions for each project.\nI tried to include what I consider the minimum necessary to understand virtual environments, their importance, and the basics of using them. The subject is very broad and there are still more tools and approaches that I did not mention here, such as Poetry, because I chose to write about the ones I am most familiar with from having used them in projects. Part of your progress is getting to know the tools, their limitations, and the contexts where they are most suitable. It is also very important to consult the documentation of each tool.\n","date":"2020-08-08","externalUrl":null,"permalink":"/2020/08/virtual-environment-python-project/","section":"Blog","summary":"In this article we will see how to use virtual environments to manage Python projects. We will see that it is possible to have a different Python version for each project, and also how packages and dependencies are controlled.","title":"Creating a virtual environment for your Python project","type":"posts"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/linux/","section":"tags","summary":"","title":"Linux","type":"tags"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/pipenv/","section":"tags","summary":"","title":"Pipenv","type":"tags"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/pyenv/","section":"tags","summary":"","title":"pyenv","type":"tags"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/venv/","section":"tags","summary":"","title":"venv","type":"tags"},{"content":"","date":"2020-08-08","externalUrl":null,"permalink":"/tags/virtual-environment/","section":"tags","summary":"","title":"virtual environment","type":"tags"},{"content":"If you started studying programming or became interested in it in recent years, you have probably heard of data science or machine learning. And you may also have heard of, or worked with, the R language, for its great power in statistical analysis, or Python, for its great versatility. Today\u0026rsquo;s article is about Anaconda, a distribution that aims to make it easier to install and manage tools for these two languages, especially for scientific applications.\nIf you prefer video, click the player below (in Portuguese). The full article follows the video.\nGetting the installer # The Anaconda project website is simple and intuitive. Here we will cover installing the Individual Edition of Anaconda, which can be downloaded here. In this article, we will install the Python 3 installer for Linux systems. At the time of writing, the name of the downloaded installer file is Anaconda3-2020.07-Linux-x86_64.sh.\nInstalling the dependencies required for the GUI # Our goal here is to install Anaconda on a Linux distribution, more specifically Linux Mint 20. According to the instructions on the website, some dependencies are needed for the applications to run in graphical mode. For Debian-based distributions, as in our case, we need to install:\nsudo apt install libgl1-mesa-glx libegl1-mesa libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6 Verifying the integrity of the installer # It is always a good idea to check that the download of an installer completed successfully, to avoid starting the installation with corrupted files that can break system dependencies. If you have never heard of checksums and cryptographic hashes, I recommend this reading.\nThe project website has a list of the hashes of all of its installer files. At the time of writing, the hash of the Python 3 installer for Linux is 38ce717758b95b3bd0b1797cc6ccfb76f29a90c25bdfa50ee45f11e583edfdbf, but you should check the website, since a new version may be available by the time you read this article.\nTo check whether the downloaded file is corrupted, we can either compare the hash of the downloaded file with the expected hash by eye, or have the system do the comparison. For the visual check, we run the sha256sum command followed by the file name and compare the result with the hash on the website:\n$ sha256sum Anaconda3-2020.07-Linux-x86_64.sh To have the system compare the two hashes:\n$ echo \u0026#34;38ce717758b95b3bd0b1797cc6ccfb76f29a90c25bdfa50ee45f11e583edfdbf Anaconda3-2020.07-Linux-x86_64.sh\u0026#34; | sha256sum --check If the output of the command is OK, the hash of the file matches the given hash.\nCreating an installation folder accessible to all users # By default, the Anaconda installer uses the user\u0026rsquo;s home folder (~/user-name). There is nothing wrong with that. However, if the machine is used by more than one user, it is sometimes better to have a single installation that serves all of them than to install it in the home folder of each user. So, before starting the installation, we will create a folder accessible to all users of the system, so that we can point the installer to it. Even if your machine is currently used only by you, I recommend keeping the installation in a system folder so that it can accommodate multiple users in the future.\nOn Linux, there is a standard hierarchy of system files (FHS, for Filesystem Hierarchy Standard) that recommends which folders to use in each situation. According to this standard, the opt folder is meant for programs that are not part of the operating system. So, we will create a folder for the installation inside opt. It is also important that this folder has the correct permissions so that the installer can write to it. So, creating an anaconda folder in opt:\nsudo mkdir /opt/anaconda \u0026amp;\u0026amp; sudo chmod ugo+w /opt/anaconda Installation # Now we can perform the installation itself. Assuming you are in the folder where the installer was downloaded:\nbash Anaconda3-2020.07-Linux-x86_64.sh -u The -u option is requested by the installer when the installation is done in an existing folder. Just follow the on-screen instructions, accept the license terms, and change the installation folder to the one we created.\nIt is also important that, at the end of the installation, you confirm the option for the installer to initialize the conda manager. This way, the installer will update the .bashrc file so that you can start the conda manager directly from the terminal.\nVerifying that the installation worked # After the installation finishes, close and reopen your terminal. If the installation succeeded, the (base) indicator will appear before the prompt:\n(base) $ This indicates that the conda base environment was activated at startup (the default behavior). You can check which Python interpreter is being used with:\n(base) $ which python /opt/anaconda/bin/python To deactivate the environment:\n(base) $ conda deactivate $ Now, the active Python interpreter is the system\u0026rsquo;s:\n$ which python /usr/bin/python Making the installation accessible to all users # Earlier, we created a system-level folder so that all users could access it. Now we need to let other users actually use the Anaconda installation in that folder. First, let\u0026rsquo;s create a user group called anaconda:\n$ sudo groupadd anaconda Now, let\u0026rsquo;s transfer ownership of the installation folder to this group:\n$ sudo chgrp -R anaconda /opt/anaconda Let\u0026rsquo;s allow only the owner, root, and the group to have read and write permission:\n$ sudo chmod 770 -R /opt/anaconda Now, add whichever users you want to this anaconda group so that they can use the installation:\n$ sudo adduser user_name anaconda In case you don\u0026rsquo;t remember, at the end of the installation we allowed the installer to initialize conda. So, the installer made changes to the .bashrc file of the user who ran the installation so that conda is initialized every time the terminal is opened. We need to apply these changes to all users. The simplest way is to put the instructions in the /etc/profile file:\n# \u0026gt;\u0026gt;\u0026gt; conda initialize \u0026gt;\u0026gt;\u0026gt; # !! Contents within this block are managed by \u0026#39;conda init\u0026#39; !! __conda_setup=\u0026#34;$(\u0026#39;/opt/anaconda/bin/conda\u0026#39; \u0026#39;shell.bash\u0026#39; \u0026#39;hook\u0026#39; 2\u0026gt; /dev/null)\u0026#34; if [ $? -eq 0 ]; then eval \u0026#34;$__conda_setup\u0026#34; else if [ -f \u0026#34;/opt/anaconda/etc/profile.d/conda.sh\u0026#34; ]; then . \u0026#34;/opt/anaconda/etc/profile.d/conda.sh\u0026#34; else export PATH=\u0026#34;/opt/anaconda/bin:$PATH\u0026#34; fi fi unset __conda_setup # \u0026lt;\u0026lt;\u0026lt; conda initialize \u0026lt;\u0026lt;\u0026lt; NOTE: It is important that you copy exactly what was added to your .bashrc file. Here I simply copied what was added to mine; you should copy what was added to yours.\nThe first time a new user opens the shell, they will need to run the conda init command and close it. That\u0026rsquo;s it: they will be able to use the installation normally the next times they open the shell, without needing that command.\nConclusion # The Anaconda installation itself is simple, but some details need to be covered for a multi-user installation. We covered those details in this article to help those who need it. We also have a more detailed article on virtual environments where we write about conda and its features. Don\u0026rsquo;t miss that article.\nIf you have questions, leave a comment. Share the article with your friends and see you in the next article.\n","date":"2020-08-01","externalUrl":null,"permalink":"/2020/08/installing-anaconda-on-linux/","section":"Blog","summary":"You have probably heard of data science or machine learning. And you may also have heard of, or worked with, the R or Python languages. Today’s article is about Anaconda, a distribution that aims to make it easier to install and manage tools for these two languages, especially for scientific applications.","title":"Installing Anaconda on Linux Mint 20","type":"posts"},{"content":"Hello! My name is Francisco Bustamante, also known as chicolucio.\nWho writes this I have a degree in Chemistry and spent more than ten years in the classroom, as a professor and researcher at Fluminense Federal University (UFF) and the Federal Institute of Rio de Janeiro (IFRJ), both in Brazil. It was while teaching that I started using Python to explain science, and Python is what led me to Data Science: I now work as a senior data scientist, focusing on recommendation systems and MLOps (machine learning operations, running models in production).\nCiência Programada brings the two together: the wish to explain, which comes from the teacher, and the habit of solving problems with code, which comes from working with data.\n10\u0026#43;\nyears in the classroom\nchemistry for over a thousand students at UFF and IFRJ, and later Python in online courses 6\u0026#43;\nyears with data\nanalyzing data with Python, from research to industry 110\narticles\npublished here since 2020 9\nseries\nsequences of articles on a theme The path, in order:\nChemistry 2006 Fluminense Federal University (UFF) Bachelor\u0026rsquo;s degree and, from 2010, master\u0026rsquo;s degree in Chemistry, publishing papers in international journals. Professor and researcher 2012 UFF General chemistry and inorganic chemistry, and projects on low-cost teaching resources. Laboratory and environment 2014 Rio de Janeiro State Environmental Institute (INEA) and UFF Environmental analyst at INEA and manager of UFF\u0026rsquo;s analytical center. Professor and researcher 2015 Federal Institute of Rio de Janeiro (IFRJ) General chemistry, inorganic chemistry and physical chemistry, and a research group on educational software with Python. Ciência Programada 2020 this site Programming, computer science and science with Python, in Portuguese, for thousands of readers a month. Data consultant 2022 NECTO Systems, for Uber Data pipelines for accounting reconciliation. Python and data science instructor 2023 Hashtag Treinamentos Teaching materials, videos and hands-on projects for online courses with thousands of students. Data scientist (consulting) 2025 NECTO and TeclaT, for Climatempo and LuizaLabs Power grid failure prediction with weather data and, currently, recommendation systems. Where to start Here I share content on Python programming, computer science, physics, mathematics, chemistry and data science, aiming to make complex topics accessible to students, professionals and anyone curious. Most of it is organized into series, which give the reader an order to follow. Some suggested paths:\nStudying computing? Start with The Language of Sets, move on to From Base Case to Infinity and The Art of Counting, and reach graphs in The Mathematics of Connections. Using Python in science or engineering? The SymPy series covers symbolic mathematics, and From Symbol to Number solves numerically what it cannot. Want to understand the machine? From Zero to Float shows how numbers are stored, and Inside the Computer opens up the rest, from digital logic to the processor. All of the site\u0026rsquo;s series are on the series page. If you prefer, browse by categories or tags.\nHow the articles are made The code runs. Each code block is executed on its own, in a clean Python environment, and the output shown in the article is checked against the real output. The facts have sources. Dates, names and historical results are checked against primary sources before publication. The formulas are verified. Every formula goes through the same renderer the reader uses before going live. The exercises have solutions. They stay collapsed, so you can try first. Contact Questions, suggestions and corrections are welcome: write to contato[@]cienciaprogramada.com.br. My full résumé and other projects are on my personal website.\nPersonal website LinkedIn GitHub ","date":"2020-07-20","externalUrl":null,"permalink":"/about/","section":"","summary":"","title":"About","type":"page"},{"content":"","date":"2020-07-20","externalUrl":null,"permalink":"/tags/open-source/","section":"tags","summary":"","title":"open source","type":"tags"},{"content":" SpeedCrunch is a program that aims to be a high-precision scientific calculator, with a simple but highly customizable interface.\nWhere to find it, license, and platforms # The program can be downloaded directly from the project website, with versions for Linux, Windows, and MacOS. Users of Linux distributions can also check whether the calculator is available in the distribution\u0026rsquo;s app store and/or repository, which is the procedure recommended by the project\u0026rsquo;s own website. For example, users of Linux Mint, the distribution I use and recommend, can download it from the app store by searching for \u0026ldquo;speedcrunch\u0026rdquo;, as in the figure below:\nSearching for SpeedCrunch in the Linux Mint app store. In general, users of Debian-based distributions can install it from the terminal if they have no app store:\nsudo apt install speedcrunch And on Fedora:\nsudo yum install speedcrunch The license is GPL and it is a free FOSS program: free and open source.\nOverview and features # The initial view of the calculator is quite simple: a text input on the last line and an empty space where a history of the latest calculations and their results will be displayed.\nThe figure below shows this screen:\nSpeedCrunch\u0026rsquo;s initial screen. Just type in the operations and press enter for the results to be displayed. Before pressing enter, partial results are displayed above the input bar.\nIn the figure, two calculations already done and a third one being typed:\nCalculations in SpeedCrunch: the history on top and the partial result above the input. The interface is available in several languages, in the menu Settings -\u0026gt; Language\u0026hellip;; here we will keep it in English.\nThe menu looks like this:\nSpeedCrunch\u0026rsquo;s language menu. Let\u0026rsquo;s face it, the color scheme and the initial screen are not very inviting. I will start by changing the calculator\u0026rsquo;s theme and increasing the font size a bit. Follow the steps in the animation below and feel free to select another theme and another font:\nImage of SpeedCrunch\u0026rsquo;s appearance menu Now it is easier to see what we are doing.\nAs usual in calculators and spreadsheets on computers, the basic operations are entered with the keys +, -, *, and /. Powers can be written with two asterisks or with the caret sign. And, like most calculators nowadays, the result of the last operation is stored in a variable called ans, which can be recalled simply by typing its name. Follow the animation:\nImage of a simple calculation in SpeedCrunch with a power and use of the ans variable A large collection of scientific constants # Now let\u0026rsquo;s move on to the areas where the calculator stands out. There are more than 100 scientific constants built into the program, with the correct number of significant figures and with the default values for International System of Units (SI). You can enable a menu that displays all the constants, grouped by category, and double-click to select one. Let\u0026rsquo;s select the Avogadro constant:\nImage of SpeedCrunch\u0026rsquo;s constants menu One of the things I like most about SpeedCrunch is the possibility of working with the keyboard only, without resorting to the mouse. This saves a lot of time. When you press Ctrl + Space, a small menu opens and you can select the constants from there. Note in the animation below that I added a panel showing the keys being pressed, to select the Avogadro constant again, but now using only the keyboard:\nImage of SpeedCrunch\u0026rsquo;s constants menu using the keyboard Much faster, don\u0026rsquo;t you agree?\nSeveral forms of numeric representation # Note in the history that the constant is stored in the program as 6.022140857e23, but the result is displayed in full. SpeedCrunch, like many other calculators, uses the letter e (the input also accepts the uppercase letter E) as a synonym for a power of 10. In science we usually work with scientific notation. So we can configure the calculator to display the results this way, following the steps below. Note that there are other display forms in the menu, more common in computer science (binary, hexadecimal, and octal). The shortcut, so you don\u0026rsquo;t have to go through the menus, is to press F5.\nImage of SpeedCrunch\u0026rsquo;s scientific notation menu Something I find quite useful is to increase the separation between digits every three digits a bit. This makes reading easier when there are many decimal places or when you need to copy the values to paper:\nImage of SpeedCrunch\u0026rsquo;s digit grouping menu Storing your own constants # I am a chemist by training, and one kind of calculation I often need to do is computing the molar mass of a given compound. It would be very useful to be able to store the result to reuse when necessary. This can be done by creating a constant and storing the value in it. Let\u0026rsquo;s calculate the molar mass of sodium chloride, NaCl, with the values present in the calculator and store it in a constant called NaCl:\nImage of a simple calculation in SpeedCrunch with creation of a constant Note that now you just have to type NaCl and the value is recognized.\nThis can be done in any situation. For example, the radius of the Earth, at the equator, is 6378 km. I can create a constant and store this value. But here comes an issue we have not worried about yet: what about units?\nWorking with units # When I talked about constants earlier, I showed the menu and that the values originally stored in the program are in SI units. So, the molar masses of chemical elements, for example, are in g/mol. By default, SpeedCrunch does not associate units with the values, leaving it to the user to know the units and take care not to perform invalid operations from the point of view of dimensional analysis. However, we can explicitly associate a unit with a given value. For example, let\u0026rsquo;s create a constant called r_terra_eq to store the Earth\u0026rsquo;s radius at the equator mentioned earlier:\nImage of a simple calculation in SpeedCrunch with creation of a constant with a unit Note that you just need to write the unit and, when necessary, the prefix, and the calculator recognizes the unit and even expresses the result in the SI base unit. What is more, the calculator stored this result as a value associated with units of dimension. This way, conversions can be made to other units that the calculator recognizes using the word in. If we want, for example, to know how many Earth radii are equivalent to 1 astronomical unit, we just do:\nImage of a simple calculation in SpeedCrunch with unit conversion That is, more than 22 thousand Earths fit in one astronomical unit. This conversion is only possible because both units are lengths. The program gives an error if an invalid conversion is requested. Try it!\nIf the excess of decimal places in the result bothers you, you can limit it by following the animation below, where I will limit it to 8 digits, more than enough for most cases:\nImage of a simple calculation in SpeedCrunch with precision adjustment And if you are bothered by having to type the units in full, you can associate the usual abbreviations with each unit. For example:\nImage of a simple calculation in SpeedCrunch with unit abbreviation The calculator was well programmed to perform unit conversions internally so as to give results that make sense. Let\u0026rsquo;s test it using the good old ideal gas equation. Remembering from science classes, we know that 1 mol of an ideal gas at Standard Temperature and Pressure (STP, 1 atm and 0 °C, which is 273.15 K) has a volume of approximately 22.4 liters. Recall:\n$$ PV = nRT \\therefore V = \\frac{nRT}{P} $$We have the value of R in the calculator, but it is in SI units. One of the few drawbacks of this calculator is that the values of the constants do not come with the associated units by default, so let\u0026rsquo;s associate the unit with the value and store the result in a constant called R:\nImage of a simple calculation in SpeedCrunch with creation of the constant R with a unit Now let\u0026rsquo;s assign to a variable called V the result of the calculation we want. Note that the values are being entered with their respective units:\nImage of a simple calculation in SpeedCrunch computing the volume of one mol of ideal gas The atmosphere is not an SI unit, but note that the calculator made the proper conversions and expressed the result in the SI unit, cubic meter. But we can easily request the conversion to liter:\nImage of a simple calculation in SpeedCrunch converting the volume to liter We got the expected result!\nFunctions # The more attentive among you may have found it odd earlier when I passed the temperature already in Kelvin to compute the volume of the ideal gas. If the calculator handles conversions, why not enter the temperature in degrees Celsius and let it convert to Kelvin?\nThis is a limitation of the calculator. Actually, to be fair, of most of the calculators available. They work well with conversions that can be done by simply multiplying by a conversion factor. However, they do not perform the operation when the conversion involves addition or subtraction. In fact, few conversions involve more than just a conversion factor, and among these few, temperature conversions stand out. Thus, the only temperature unit the calculator recognizes is Kelvin, as explained in the calculator\u0026rsquo;s user guide available on the project website.\nBut nothing we cannot solve. And to solve it we will use the concept of function.\nFirst, let\u0026rsquo;s see the functions already present in the calculator. There are dozens, and they can be accessed through the menu or directly by typing the name:\nImage of SpeedCrunch\u0026rsquo;s functions menu Note in the animation above that we can browse the existing functions through the menu. If we double-click a function, it goes to the input line. We can also type the name of the function and use it directly; in the animation we use the sqrt function to compute the square root of 9.\nDefining your own functions # Of course we can define our own function. So let\u0026rsquo;s define a function that computes the temperature in Kelvin from a temperature in Celsius. We know that to convert from Celsius to Kelvin we just add 273.15 to the Celsius value. So, let\u0026rsquo;s define a function T_kelvin(T_celsius) = (T_celsius + 273.15) kelvin. The \u0026ldquo;kelvin\u0026rdquo; after the operation in parentheses makes the result come out with the associated unit, since I plan to use the result later. Let\u0026rsquo;s see:\nImage of a simple calculation in SpeedCrunch with the definition of the function T_kelvin Test it by passing the value 25 as the argument. The answer must be 298.15 K.\nWe can also create functions with more than one argument, just by separating them with ;. Thus, the whole calculation done earlier to compute the volume of one mol of an ideal gas can be simplified if we create a function that takes the pressure, the number of moles, and the temperature, with their respective units. Remember that we already defined R in this tutorial with its SI unit, so we will reuse it in the definition of our function, which we will call V_ideal:\nImage of a simple calculation in SpeedCrunch with the definition of the function V_ideal Note in the animation that we can even pass the function we created to convert temperature from Celsius to Kelvin, since we defined the function with a unit.\nThe function stays saved for later use whenever you need it. User-created constants, variables, and functions can be consulted:\nImage of SpeedCrunch\u0026rsquo;s menu for consulting user creations Overall evaluation, final thoughts, and how to contribute to the project # SpeedCrunch is an excellent scientific calculator for the desktop, standing out from the others for handling units and for having visual highlighting of reserved words, functions, and variables (syntax highlighting). It is quite customizable, with many more options than the ones shown here.\nIt is excellent for students who want to check whether their calculations involving conversions are correct. And, as a teacher, I often use it in class when a projector is available to do calculations with the students, especially because of how easy it is to discuss operations with units, scientific notation, and even significant figures.\nIn my use I only miss that the stored constants are not already entered with their units by default and that there is no way to export the constants and functions created (but this only affects those who use more than one computer, at home and at work, for example). It would also be great to have a mobile version.\nHowever, such limitations are tiny. And we must remember that this is a free program, the project is maintained by donations, and you can contribute through the project website. Since it is open source, you can also contribute with fixes, questions, and suggestions in the project repository.\nSee you next time!\n","date":"2020-07-20","externalUrl":null,"permalink":"/2020/07/speedcrunch-high-precision-scientific-calculator/","section":"Blog","summary":"SpeedCrunch is a high-precision scientific calculator with a simple but highly customizable interface. It has a large collection of constants and functions. Free and open source.","title":"SpeedCrunch - a high-precision scientific calculator","type":"posts"},{"content":"Ciência Programada © 2020 by Francisco Lucio de Schneider Bustamante is licensed under CC BY-NC-SA 4.0.\nWe accept no liability for financial or non-financial loss or damage of any kind resulting from the use of the information on our site, in its content or comments, in our discussion forum, in our newsletter, or in our posts on social media (Facebook, Twitter, YouTube, etc.).\nViews expressed by users in the site\u0026rsquo;s comments section or on social media do not necessarily reflect those of Ciência Programada or the people who run it.\nBy accessing our site, forum or social media pages, you acknowledge and agree to the license and terms above.\nSee also our Privacy Policy.\n","externalUrl":null,"permalink":"/legal/","section":"","summary":"","title":"Legal Information","type":"page"},{"content":"Last updated: October 31, 2026.\nWho we are # Ciência Programada is a website about programming applied to science. Learn more about who produces the content on the About page.\nThe site has two addresses: https://cienciaprogramada.com.br, in Portuguese, and https://cienciaprogramada.com, in English. This policy applies to both.\nThe controller of personal data is the site\u0026rsquo;s author, Francisco Lucio de Schneider Bustamante, in Brazil. The contact address is contato[@]cienciaprogramada.com.br.\nSee also our Legal Information.\nWhat personal data we collect and why # Reading the site does not require registration. Personal data is collected in only three situations: when you comment on an article, through the tools that measure traffic, and through ads. Each one is described below.\nThis site does not collect sensitive personal data. Providing data is not mandatory: without it, you simply cannot comment. The site\u0026rsquo;s author does not make automated decisions about you; Google selects personalized ads.\nLegal bases # We process personal data under Brazil\u0026rsquo;s General Personal Data Protection Law (LGPD) and, for visitors from the European Economic Area (EEA), the European Union\u0026rsquo;s General Data Protection Regulation (GDPR).\nProcessing Legal basis Publishing comments and keeping a commenter account Consent (LGPD, art. 7, I; GDPR, art. 6(1)(a)) Partial IP address, session records and comment area statistics Legitimate interest in protecting the site from abuse (LGPD, art. 7, IX; GDPR, art. 6(1)(f)) Technical access data recorded by Cloudflare Legitimate interest in delivering and protecting the site (LGPD, art. 7, IX; GDPR, art. 6(1)(f)) Google Analytics and Google AdSense, for visitors from the EEA, the UK and Switzerland Consent, given in the message displayed by Google (GDPR, art. 6(1)(a)); without it, no cookies are set Google Analytics and Google AdSense, for other visitors Legitimate interest in measuring use of the site and supporting it with ads (LGPD, art. 7, IX) You can withdraw your consent at any time, without affecting anything done before the withdrawal. See Your rights.\nComments # Comments use Comentario, an open-source program installed on a server maintained by the site\u0026rsquo;s author. Comments are stored on that server, not on a third-party platform.\nThe comment area only loads when you scroll close to it. Someone who reads an article without reaching the end has no contact with the comment server.\nEvery comment goes through approval before it appears. Once approved, the text and the writer\u0026rsquo;s name are visible to any visitor.\nWays to comment # There are two ways to comment, and the data stored depends on which one you choose.\nWithout registering. You provide only a name, which can be a nickname. We do not ask for an email address or create an account.\nWith your Google or GitHub account. You log in on Google\u0026rsquo;s or GitHub\u0026rsquo;s own page, and your password never reaches our server. We receive your name, email address, profile picture and an account identifier from these services, and use them to create a commenter account. Your email address is not displayed on the site; it identifies the account and is used to notify you of replies, if you ask for those notifications. Use of these services is subject to Google\u0026rsquo;s privacy policy and GitHub\u0026rsquo;s.\nYou cannot create an account with an email address and password: we do not store commenters\u0026rsquo; passwords.\nWhat is stored # The comment\u0026rsquo;s text, its date and the name provided. Part of the commenter\u0026rsquo;s IP address. Only the first two blocks of the address are stored (for example, 203.0.x.x), which does not identify a specific computer. When available, the country estimated from the address is also stored. For those who comment with Google or GitHub: name, email address, profile picture, account identifier on the service, and the dates the account was created and last accessed. For each session of a logged-in user: date, part of the IP address, browser, operating system and device type. Comment area view statistics: page visited, date, part of the IP address, browser, operating system and device type. These records are deleted after 30 days. If the account has no profile picture, the comment server queries Gravatar with a code derived from the email address, to check whether an image is associated with it.\nWe do not use external spam detection services: the site\u0026rsquo;s author reviews comments.\nHow to change or delete your commenter data # You can view, correct or delete your data at any time.\nIf you commented with Google or GitHub, you can edit or delete your comments in the comment area itself while logged in. In your profile menu, in the same area, you can change your account details and delete the account.\nIf you commented without registering, no account is linked to the comment, so you cannot edit or delete it yourself. Write to the site\u0026rsquo;s author with the article and the name you used, and the comment will be removed.\nFor any other request, such as finding out what data of yours is stored or asking for all of it to be deleted, use the contact details on the About page.\nTo revoke the access granted at login, use the security settings of your Google or GitHub account.\nContact forms # There are currently no contact forms on this site.\nCookies # The site\u0026rsquo;s pages are static and do not set their own cookies for reading. Cookies and data stored in your browser come from three sources.\nComments. When the comment area loads, the comentario_commenter_session cookie keeps your session for up to 30 days, so that you stay logged in between visits. The browser also stores locally the draft of the comment you are writing, the name used to comment without registering, and the display order you chose. This data stays on your device and can be deleted in your browser settings. Traffic measurement. Google Analytics uses cookies to distinguish visitors. See the Analytics section. Ads. Google AdSense uses cookies to display and measure ads. See the Ads section. Embedded content from other websites # Articles on this site may include embedded content such as videos, images and articles. Embedded content from other websites behaves exactly as if the visitor had visited that website.\nThese websites may collect data about you, use cookies, embed additional third-party tracking and monitor your interaction with the embedded content, including when you have an account and are logged in to that website.\nAnalytics # This site uses Google Analytics to measure traffic. Google Analytics associates the information it collects about visits with the Google Accounts of users who have logged in and allowed this association for ad personalization. This information includes the end user\u0026rsquo;s location, search and YouTube history, and data from Google\u0026rsquo;s partner sites. It is also used to provide aggregated, anonymous insights into how visitors behave across devices.\nThese features are subject to Google\u0026rsquo;s privacy policy.\nFor visitors from the EEA, the UK and Switzerland, Google Analytics only sets cookies after consent is given in the message displayed by Google.\nAds # This site displays ads from Google AdSense. Google and its partners use cookies to serve ads based on your visits to this and other websites. You can turn off personalized advertising in Google\u0026rsquo;s ad settings and learn more about how Google uses information from partner sites.\nVisitors from the EEA, the UK and Switzerland see a consent message from Google before ads are personalized.\nWho we share your data with # We do not sell or transfer personal data to third parties.\nApproved comments are public: any visitor can read the text and the writer\u0026rsquo;s name. The email address and the other data described under Comments are restricted to the site\u0026rsquo;s author.\nWhere we send your data # Access data is sent to Google Analytics, and ads are served by Google AdSense. If you comment with your Google or GitHub account, the login goes through those services. A code derived from the commenter account\u0026rsquo;s email address may be sent to Gravatar to look up the profile picture. The site\u0026rsquo;s pages are delivered through Cloudflare\u0026rsquo;s network, which records technical access data, such as the IP address, to operate and to protect itself from attacks. How long we keep data # Comments: until they are removed, by you or at your request. Commenter account: until you delete it or ask for it to be deleted. Comment area view statistics: 30 days. Comment session cookie: up to 30 days. Google Analytics, Google AdSense and Cloudflare: for the periods set in those services. Your rights # You can ask us to confirm whether we process your data, and for access to it and its correction, deletion or portability. You can also restrict or object to its processing and withdraw any consent you have given.\nTo do so, write to contato[@]cienciaprogramada.com.br. We will reply within the period set by the applicable law. For comments, you can also follow the steps under How to change or delete your commenter data.\nYou also have the right to complain to a data protection authority: in Brazil, the National Data Protection Authority (ANPD); in the EEA, the authority of the country where you live or work, or where the alleged infringement took place.\nContact information # If you have questions, write to contato[@]cienciaprogramada.com.br or use the platforms listed on the author\u0026rsquo;s About page.\nAdditional information # How we protect your data # We do not collect sensitive data or store commenters\u0026rsquo; passwords.\nThe connection to the site and to the comment server is protected by encryption (HTTPS). The comment database is not accessible from the internet, and the administration area requires authentication.\n","externalUrl":null,"permalink":"/privacy-policy/","section":"","summary":"","title":"Privacy Policy","type":"page"}]