The theoretical foundation and the fundamental algorithms for nonlinear optimization are studied and applied to supervised learning models, including nonlinear regression, logistic regression, support ...