Data Jungler

另一種稱為邏輯回歸 (Logistic Regression)的回歸模型用於描述某類別 (Categorical Variable)與獨立變量 (Independent Variables)之間的關係，並估計出現某類別的概率。 邏輯回歸中的輸出變量是分類而不是間隔。 可能讀者們早已聽過機器學習的Classification，它是Data Mining和Pattern Recognition中常見的應用之一。透過Logit Function的轉換，目標變量的類型變成了Categorical，不再是Numeric。 為了讓初學者容易理解邏輯回歸的精要，在本文中，會利用只有兩個類別的目標變數會作為例子（兩個類別為“是”或“否”）。 公式: $$ln(\frac{p}{1-p}) = w_0 + w_1x_1 + w_2x_2+...+w_nx_n=\sum_{i=1}^{m}{w_ix_i}=w^Tx$$ 當中的$p$為得到“是”的機率，透過一些轉換，我們會獲得新的公式（如下）。假設我們已經知道所有自變量, $x$和每個自變量的比重$w$，我們便能計算出得到“是”的機率。至於“否”的機率，明顯地是$1 - p$ $$p=\frac{1}{1+e^{-w^Tx}}$$ 邏輯回歸的Cost function的公式定義在下面，這其實由統計學上的Maximum Likelihood Estimation衍生出來，同樣地，目的是把Cost function的減至最少，從而獲得相應的比重, $w_i$。 $$J(w) = -\frac{1}{m}[\sum_{j=1}^{m}{(y^{(j)}log(h_w(x^{(j)}) + (1-y^{(j)})(log(h_w(x^{(j)}))}]$$ 從機器學習的角度出發，邏輯回歸運算的要求較低，易於實現和解釋。 然而，面對一些非線性關係的自變量，違反了模型本身的假設，其表現亦會減弱，即是更容易錯誤地分類。 在Python實現 1. 安裝Scikit-Learn 參考 先前的文章 安裝用作機器學習的Package。 2. 數集 數據 及可視化 假設我們有一組客戶的數據，包括他們的個人資料，消費量和今年有沒有續會（ 按這裡下載 ）。 我們的目標是根據這些歷史數據 (Training Data)預測一些新加...

Another regression model called Logistic regression is used to describe the relationship between the categorical dependable variable and independable variables with logistic function and estimate the probabilities of binary response. The output variable in logistic regression is categorical rather than interval. You might have heard the terms, Classification, which is one of common applications in data mining or automatic pattern recognition. We can classify the binary, ordinal or nominal target variables by performing Logistic Regression. In this tutorial, only binary target variables ("Yes", "No") is taken as example of classification in application since it is less complicated for beginner to get a grip on. Formula: $$ln(\frac{p}{1-p}) = w_0 + w_1x_1 + w_2x_2+...+w_nx_n=\sum_{i=1}^{m}{w_ix_i}=w^Tx$$ The probability of being "Yes", $p$ can be expressed as below by modifying above formula. If the probability exceeds the threshold (generally 0.5), the...

Recently, I have attended the Open Data Hackathon 2016 organized in Hong Kong, and join a team with others from different backgrounds. It is really a great opportunity to learn from the experienced, how they process the data with different kinds of exploration techniques, visualize and present the analytic results. The most valuable is to know a new technique, Autoregression to apply on the time series data. Previously, I usually adopt the multiple linear regression on the non-time-series data to investigate the factors effecting on dependent variables. Additionally, our team was focusing on the air pollution in Hong Kong and wished to figure out how the containment gas was dependent of policy, sources and environment so as to think a series of improvement way or policy to alleviate the problems. Of course, there were only limited time and resources, it was ideal to finish within one day. In order to explore such data, we selected one of the polluted air, Sulphur dioxide to study. Ho...

最近，我在一個在杜雷斯專頁的貼文中（利申：沒有收取廣告費，要給我是歡迎的，笑）看到了有一個決策樹(Decision Tree)，我本來一笑而之，可是再想一想，用直覺去判斷這個樹的設計，也好像是挺合理的。突然，我的靈感就跑出來，如果用數據去建模(Modeling)的話，結果會一樣嗎？這個無聊的想法也驅使我做更無聊的事，就是把分類決策樹(Classification Tree)建造出來 LOL，看看朋友們的想法是否接近這幅圖所表達的~ 如果自己沒有膽量，又想了解女神或者男神對你的感覺，這棵樹應該可以幫助你，因為它能大概預測到男神或女神對你有沒有好感！ 剛好筆者正在學習使用R，所以分析和建模的那部分這一次就交給R，來當作練習吧～所以對不起了，Python大大~_~   來源：杜雷斯Facebook專頁 首先，我認清自己的目標和方向 目標變數 (Target Variables)： 對異性的感覺，值有以下3個： $y_1$: 有好感的（好感） $y_2$: 沒有好感，但覺得是好人 （好人） $y_3$:  完全沒有好感（BYE~） 輸入變數(Input Variables)： $x_1$: 外表, Appearance（帥vs不帥, 美女vs不好看） $x_2$: 交流, Chatable（聊得來，聊不來） 為了獲得數據，最直接了當就是去做訪問，而訪問對象當然就是自己的朋友們 （還好沒有傷害到友情） ，去了解他們對自己的朋友X先生/Y小姐們的感覺。 想直接看結果，直接拉去 ===分隔線=== 吧～ 於是我設計幾個問題, 來 刁難 訪問我幾位的朋友們： 1a. 你覺得X先生/Y小姐外表吸引嗎？     對／不對 1b. 你覺得X先生/Y小姐的樣子帥嗎？     對／不對 1c. 你覺得X先生/Y小姐的身材好嗎？     對／不對 2a. 你跟X先生/Y小姐聊得來嗎？               對／不對 2b. 你會想繼續跟X先生/Y小姐...

線性回歸 (Linear Regression) 應用於不同的方面以達到不同的目的。 描述自變量Independent Variables（$ x_i $）和因變量Dependent Variables（$ y $）之間的關係的方法，或者通過輸入（$ x_i $）預測目標變量（$ y $）。 還有助於我們了解在更改一個或多個獨立變量時對因變量變化的影響。 公式: $$y = w_0+w_1x_1+w_2x_2+ ... +w_nx_n$$ 也可以表示為: $$h_w(x)=\sum_{i=1}^{m}{w_ix_i}=w^Tx$$ Cost Function定義為$h_w(x^{(j)})$和$y^{(j)}$之間的差的平方和。 統計學上，被稱為平方誤差(Sum of Square Error, SSE)。 為了使直線符合$n$數據點，Cost Function / SSE需要最小化以實現優化目標，這過程又稱OLS Estimation(Ordinary least squares Estimation)。 直觀地想，預測值(Predicted value)和實際目標(Actual value)值之間的差異越小，預測模型給出的結果越接近實際值，也暗示這個模型的愈準確，誤差值亦愈小，$R^2$表示線性回歸模型解釋數據的能力。愈高的$R^2$意味著模型愈有能力解釋現有的數據。 它還量度總變異, $SST$和回歸平方誤差的和, $SSR$的比例。 $R^2=\frac{SSR}{SST}$, where $0\leq R^2\leq 1$ 但是，線性回歸面對著非線性關係的變數，不如其他模型那樣精確，如神經網絡，始終它假設輸入和輸出的線性關係。 線性回歸的另一個問題是它對Outlier（主要是Influence Point) 的敏感性，會影響預測結果的準確性。 在我們訓練模型之前，可以在某些圖表中或者統計結果中被識別，把Outlier踢出來解決這個問題。 In previous chapter , I have introduced the overall process of data mining. Some of you might not understand well. It should be fine since the s...

No doubt that Linear Regression are competent to be applied to different aspects and purposes. It can be an approach to describe the relationship between the independent variables ($x_i$) and dependable variable ($y$), or predict the target variables ($y$ ) with the inputs ($x_i$). It also helps us to understand how much effect on change of dependent variable, when changing one or more independent variables. Formula: $$y = w_0+w_1x_1+w_2x_2+ ... +w_nx_n$$ It is also presented as: $$h_w(x)=\sum_{i=1}^{m}{w_ix_i}=w^Tx$$ Cost Function is defined as sum of square of difference between $h_w(x^{(j)})$ and $y^{(j)}$. In statistics, it is called sum of square error, SSE. In order to fit the straight line to $n$ data points, cost function / SSE needs to be minimized to achieve the optimization goal.  $$J(w)=\sum_{j=1}^{m}{h_w(x^{(j)})-y^{(j)})^2}$$ Think it intuitively, the less difference between predicted and actual target values, the predictive model is giving the result which...

Scikit-Learn is a famous package of machine learning in Python, that are contributed by the community. Most of the processes involving data mining including data processing, model training and validation and assessment are available in this module. Some highlighted algorithms of Regression, Decision Tree, Clustering are allowed to satisfy our analytic needs. Recently, I am glad to find a new class of Artificial Neural Network for supervised learning in the new version. Really cannot wait for the latest version of scikit-learn! So, install Scikit-Learn first! Mac Open the terminal and type: pip install scikit-learn Window Open the CMD: pip install scikit-learn Ubuntu Open the terminal and type: pip install --user --install-option="--prefix=" -U scikit-learn Let's enjoy the journey of data mining in data jungle with Scikit-learn!