/admin

O'reilly Feature Engineering For Machine Learning Pdf

Machine learning and
data mining
  • Ensembles

  • DBSCAN
  • Graphical models
  • RNN
  • Convolutional neural network

Feature engineering is the process of using domain knowledge to extract features from raw data via data mining techniques. These features can be used to improve the performance of machine learning algorithms. Feature engineering can be considered as applied machine learning itself [1].

Features[edit]

Feature engineering is generally the section that gets left out of machine learning books, but it's also the most critical part in practice. Ted Dunning explores techniques, a few well known, but some rarely spoken of outside the institutional knowledge of top teams, including how to handle categorical inputs, natural language, transactions, and more in the context of machine learning.

A feature is an attribute or property shared by all of the independent units on which analysis or prediction is to be done. Any attribute could be a feature, as long as it is useful to the model.

The purpose of a feature, other than being an attribute, would be much easier to understand in the context of a problem. A feature is a characteristic that might help when solving the problem.[2]

Importance[edit]

Features are important to predictive models and influence results [3].

It is asserted that feature engineering plays an important part of Kaggle competitions [4] and machine learning project's success or failure [5].

Laview app setup Configure client settingsUse the to configure enrollment for Mac computers. ImportantTo deploy the client to devices running macOS Sierra, correctly configure the Subject name of the management point certificate. Re-enroll the Mac computer before the installed certificate expires. You can't use custom client settings. For example, use the FQDN of the management point server.

Vidbox

Process[edit]

The feature engineering process is:[6]

  • Brainstorming or testing features;[7]
  • Deciding what features to create;
  • Creating features;
  • Checking how the features work with your model;
  • Improving your features if needed;
  • Go back to brainstorming/creating more features until the work is done.

Relevance[edit]

A feature could be strongly relevant (i.e., the feature has information that doesn't exist in any other feature), relevant, weakly relevant (some information that other features include) or irrelevant.[8] Even if some features are irrelevant, having too many is better than missing those that are important. Feature selection can be used to prevent overfitting.[9]

Feature explosion[edit]

Feature explosion can be caused by feature combination or feature templates, both leading to a quick growth in the total number of features.

  • Feature templates - implementing feature templates instead of coding new features
  • Feature combinations - combinations that cannot be represented by the linear system

Feature explosion can be stopped via techniques such as: regularization, kernel method, feature selection.[10]

Automation[edit]

Automation of feature engineering is a research topic that dates back to at least the late 1990s.[11] The academic literature on the topic can be roughly separated into two strings: First, Multi-relational decision tree learning (MRDTL), which uses a supervised algorithm that is similar to a decision tree. Second, more recent approaches, like Deep Feature Synthesis, which use simpler methods.[citation needed]

Multi-relational decision tree learning (MRDTL) generates features in the form of SQL queries by successively adding new clauses to the queries.[12] For instance, the algorithm might start out with

The query can then successively be refined by adding conditions, such as 'WHERE t1.charge <= -0.392'.[13]

However, most of the academic studies on MRDTL use implementations based on existing relational databases, which results in many redundant operations. These redundancies can be reduced by using tricks such as tuple id propagation.[14][15] More recently, it has been demonstrated that the efficiency can be increased further by using incremental updates, which completely eliminates redundancies.[16]

In 2015, researchers at MIT presented the Deep Feature Synthesis algorithm and demonstrated its effectiveness in online data science competitions where it beat 615 of 906 human teams.[17][18] Deep Feature Synthesis is available as an open source library called Featuretools.[19] That work was followed by other researchers including IBM's OneBM[20] and Berkeley's ExploreKit.[21] The researchers at IBM stated that feature engineering automation 'helps data scientists reduce data exploration time allowing them to try and error many ideas in short time. On the other hand, it enables non-experts, who are not familiar with data science, to quickly extract value from their data with a little effort, time, and cost.'[citation needed]

See also[edit]

References[edit]

  1. ^'Machine Learning and AI via Brain simulations'. Stanford University. Retrieved 2019-08-01.
  2. ^'Discover Feature Engineering, How to Engineer Features and How to Get Good at It - Machine Learning Mastery'. Machine Learning Mastery. Retrieved 2015-11-11.
  3. ^'Feature Engineering: How to transform variables and create new ones?'. Analytics Vidhya. 2015-03-12. Retrieved 2015-11-12.
  4. ^'Q&A with Xavier Conort'. kaggle.com. 2013-04-10. Retrieved 12 November 2015.
  5. ^Domingos, Pedro (2012-10-01). 'A few useful things to know about machine learning'(PDF). Communications of the ACM. 55 (10): 78–87. doi:10.1145/2347736.2347755.
  6. ^'Big Data: Week 3 Video 3 - Feature Engineering'. youtube.com.
  7. ^Jalal, Ahmed Adeeb (January 1, 2018). 'Big data and intelligent software systems'. International Journal of Knowledge-based and Intelligent Engineering Systems. 22 (3): 177–193. doi:10.3233/KES-180383 – via content.iospress.com.
  8. ^'Feature Engineering'(PDF). 2010-04-22. Retrieved 12 November 2015.
  9. ^'Feature engineering and selection'(PDF). Alexandre Bouchard-Côté. October 1, 2009. Retrieved 12 November 2015.
  10. ^'Feature engineering in Machine Learning'(PDF). Zdenek Zabokrtsky. Archived from the original(PDF) on 4 March 2016. Retrieved 12 November 2015.
  11. ^Knobbe, Arno J.; Siebes, Arno; Van Der Wallen, Daniël (1999). 'Multi-relational Decision Tree Induction'(PDF). Principles of Data Mining and Knowledge Discovery. Lecture Notes in Computer Science. 1704. pp. 378–383. doi:10.1007/978-3-540-48247-5_46. ISBN978-3-540-66490-1.
  12. ^'A Comparative Study Of Multi-Relational Decision Tree Learning Algorithm'. CiteSeerX10.1.1.636.2932.Cite journal requires journal= (help)
  13. ^Leiva, Hector; Atramentov, Anna; Honavar, Vasant (2002). 'Experiments with MRDTL – A Multi-relational Decision Tree Learning Algorithm'(PDF).Cite journal requires journal= (help)
  14. ^Yin, Xiaoxin; Han, Jiawei; Yang, Jiong; Yu, Philip S. (2004). 'CrossMine: Efficient Classification Across Multiple Database Relations'. Proceedings of the 20th International Conference on Data Engineering. doi:10.1109/ICDE.2004.1320014.
  15. ^Frank, Richard; Moser, Flavia; Ester, Martin (2007). 'A Method for Multi-relational Classification Using Single and Multi-feature Aggregation Functions'. Knowledge Discovery in Databases: PKDD 2007. Lecture Notes in Computer Science. 4702. pp. 430–437. doi:10.1007/978-3-540-74976-9_43. ISBN978-3-540-74975-2.
  16. ^'How automated feature engineering works - The most efficient feature engineering solution for relational data and time series'. Retrieved 2019-11-21.[promotional source?]
  17. ^'Automating big-data analysis'.
  18. ^Kanter, James Max; Veeramachaneni, Kalyan (2015). 'Deep Feature Synthesis: Towards Automating Data Science Endeavors'. IEEE International Conference on Data Science and Advanced Analytics. doi:10.1109/DSAA.2015.7344858.
  19. ^'Featuretools An open source framework for automated feature engineering Quick Start'. www.featuretools.com. Retrieved 2019-08-22.
  20. ^Hoang Thanh Lam; Thiebaut, Johann-Michael; Sinn, Mathieu; Chen, Bei; Mai, Tiep; Alkan, Oznur (2017). 'One button machine for automating feature engineering in relational databases'. arXiv:1706.00327.Cite journal requires journal= (help)
  21. ^'ExploreKit: Automatic Feature Generation and Selection'(PDF).

Further reading[edit]

  • Boehmke, Bradley; Greenwell, Brandon (2019). 'Feature & Target Engineering'. Hands-On Machine Learning with R. Chapman & Hall. pp. 41–75. ISBN978-1-138-49568-5.
  • Zheng, Alice; Casari, Amanda (2018). Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists. O'Reilly. ISBN978-1-4919-5324-2.
  • Zumel, Nina; Mount, John (2020). 'Data Engineering and Data Shaping'. Practical Data Science with R (2nd ed.). Manning. pp. 113–160. ISBN978-1-61729-587-4.
Retrieved from 'https://en.wikipedia.org/w/index.php?title=Feature_engineering&oldid=941345537'