Wide and Deep Learning Models Explained in BigQuery
Written on
Chapter 1 Overview of Wide and Deep Models
When exploring Wide and Deep Learning Models, one might wonder what they truly entail and how they can be applied. These models consist of two primary elements:
- Memorization Component: This is represented by a linear model.
- Generalization Component: This is realized through a neural network.
Additionally, these two components can interact through a cross-product, allowing for enhanced functionality. A common application of Wide and Deep Learning Models is in recommendation systems. For instance, if you order a Big Mac at McDonald's, it's likely you will also receive a side of fries or a Coke due to these intelligent systems.
Theoretical Basis
A wide linear model serves the purpose of memorization, while a deep neural network focuses on generalization. By merging the advantages of both, we move closer to achieving optimal results. — Heng-Tze Cheng [1]
Illustration of a Wide and Deep Model — Source: Google AI Blog [1]
For a comprehensive understanding of how these models function together, I recommend checking out the Google AI Blog, which has greatly enhanced my grasp of the subject.
Section 1.1 Implementing Wide and Deep Models in BigQuery
In BigQuery, you can effortlessly create models using straightforward SQL commands. This approach signifies a new paradigm known as "Bring Compute to the Data." Below is a basic blueprint for constructing such a model:
CREATE MODEL project_id:mydataset.mymodel
OPTIONS(MODEL_TYPE='DNN_LINEAR_COMBINED_CLASSIFIER',
ACTIVATION_FN = 'RELU',
BATCH_SIZE = 16,
DROPOUT = 0.1,
EARLY_STOP = FALSE,
HIDDEN_UNITS = [128, 128, 128],
INPUT_LABEL_COLS = ['mylabel'],
LEARN_RATE=0.001,
MAX_ITERATIONS = 50,
OPTIMIZER = 'ADAGRAD')
AS SELECT * FROM project_id:mydataset.mytable;
For more details and further parameters, you can refer to the sources below (Google Source). I must confess that I have yet to create my own model, as I have not encountered an appropriate dataset — unless you happen to be a data scientist at McDonald's! Nevertheless, I find this concept thrilling, and perhaps someone out there has an interesting application that can be implemented in Google BigQuery.
As this feature is relatively new, it may not be accessible in all regions. If you have a dataset that could be of interest to experiment with, please reach out.
Chapter 2 Video Resources
To further enhance your understanding, here are two insightful videos:
The first video titled "How to use LLMs in SQL Statements with Big Query ML" provides a clear overview of how to effectively integrate Large Language Models into SQL queries.
The second video, "How to accelerate machine learning development with BigQuery ML," delves into methods for speeding up the machine learning development process using BigQuery ML.
Sources and Further Readings
[1] Heng-Tze Cheng (Google AI Blog), Wide & Deep Learning: Better Together with TensorFlow (2016)
[2] Google, The CREATE MODEL statement for Wide-and-Deep models (2022)