猫史档案馆

除了梯度下降法以外，还有没有办法可以拟合数据呢？当然有，今天就给大家介绍一种新的方法：最小二乘法。

用这种方法算起来非常快，不同于梯度下降使用计算机的蛮力对抗精准。最小二乘法更像是人类的艺术，而不是一种ai算法。

假设现在我们有两个向量A：【1,3】。B【3,1】。

现在我们要在A的直线上找到一个点使它到B的距离最短。即使得【1*x,3*x】与【3,1】的差的平方最小。【1*x,3*x】我们将简写成Ax。

然后我们就得到了这个问题的损失函数（Ax-B）²

拆开后：A²x²+B²-2Abx。我们将这个函数求导（过程不再给出，用之前我教你们的方法自己算）得到：2xA²-2AB。这是一个凹函数，可以脑补一下它大概长这样：

在导数为0时，这个函数的值最小（处在谷底）。所以：

2xA²-2AB = 0

2xA² = 2AB

xA² = AB

将左边乘上A²的逆，A²是float，所以与逆相乘为1,所以抵消：

定义：求逆=inv

x = AB * inv（A²）

然后我们就得出了x的值，最后带回原式：

P = AB * inv（A²）* A

然后我们就可以利用这个结论x = AB * inv（A²）来解决复杂的问题了！

 

写了一个最小二乘框架，非常方便，给定条数据和标签和要拟合的函数的次数就可以拟合，有兴趣的可以看看：

import numpy as np
import warnings

class least_square():
    def __init__(self):
        self.number = 2
        self.data = []
        self.datax = []
        self.label = []
        self.labely = []
        self.p = 0
        self.prediction = []
    def fit_function_number(self,x):
        if x < 2 or int(x) != x:
            raise Exception("fit_function's Number can't be this:(Int and larger than 1)", x)
        self.number = x
    def input_data(self,data,label):
        self.datax = data
        self.labely = label
        data_memroy = []
        xdata_memroy = []
        for a in range(len(data)):
            xdata_memroy = []
            for b in range(self.number):
                xdata_memroy.append(data[a]**b)
            data_memroy.append(xdata_memroy)
        self.data = np.array(data_memroy)
        self.label = np.array([label]).T
    def fit(self):
        self.p = np.linalg.inv(self.data.T.dot(self.data))*(self.data.T.dot(self.label))
        prediction = np.zeros((len(self.p),self.number))
        for a in range(len(self.p)):
            prediction += self.p[a]
        self.prediction = prediction[0]
        warnings.warn('Result is from b-->kn')
        return prediction[0]
if __name__ == '__main__':
    least_square = least_square()
    x = [1,2,3,4,5]
    y = [7,13,28,40,68]
    least_square.fit_function_number(4)
    least_square.input_data(x,y)
    a = least_square.fit()
测试：

比梯度下降快很多，效果不错！

import numpy as np
import warnings
from matplotlib import pyplot as plt
class least_square():
    def __init__(self):
        self.number = 2
        self.data = []
        self.datax = []
        self.label = []
        self.labely = []
        self.p = 0
        self.prediction = []
    def fit_function_number(self,x):
        if x < 2 or int(x) != x:
            raise Exception("fit_function's Number can't be this:(Int and larger than 1)", x)
        self.number = x
    def input_data(self,data,label):
        self.datax = data
        self.labely = label
        data_memroy = []
        xdata_memroy = []
        for a in range(len(data)):
            xdata_memroy = []
            for b in range(self.number):
                xdata_memroy.append(data[a]**b)
            data_memroy.append(xdata_memroy)
        self.data = np.array(data_memroy)
        self.label = np.array([label]).T
    def fit(self):
        self.p = np.linalg.inv(self.data.T.dot(self.data))*(self.data.T.dot(self.label))
        prediction = np.zeros((len(self.p),self.number))
        for a in range(len(self.p)):
            prediction += self.p[a]
        self.prediction = prediction[0]
        warnings.warn('Result is from b-->kn')
        return prediction[0]
    def predict(self,x):
        result = 0
        for i in range(self.number):
            result += self.prediction[i] * (x**i)
        return result
if __name__ == '__main__':
    least_square = least_square()
    x = [1,2,3,4,5]
    y = [7,13,28,40,68]
    least_square.fit_function_number(20)
    least_square.input_data(x,y)
    a = least_square.fit()
    plt.scatter(x,y)
    x1 = np.linspace(1,6,100)
    y1 = least_square.predict(x1)
    plt.plot(x1,y1)
    plt.show()