You can use Google BigQuery to set up quick, baseline models for predictions and custom machine learning models. It’s easy to build models directly inside BigQuery using simple SQL in a fraction of the time it takes with other methods.
BigQuery ML Acts as the Perfect Baseline Strategy
Sometimes you want to do custom modeling and other times you simply want to develop a quick baseline model for predictions. BigQuery ML works as a great baseline strategy. There’s no need to move data to or from anywhere. You simply execute SQL right inside the BigQuery UI, delivering an understanding of what results and accuracy you can expect.
BigQuery ML offers a full range of models to fit your needs, including:
- Linear regression
- Classification or logistic regression
- K-means clustering
- Matrix factorization
You can import TensorFlow models to use in BigQuery ML, as well.
BigQuery ML Use Case Example
Say you want to segment your customer base. Start by building a k-means clustering model using BigQuery ML. Or, you can select from the other models depending on your needs. You’ll need to understand the type of model you’re looking for to efficiently use BigQuery ML.
Once you choose your model, select your hyperparameters or configurable options for the outcome. You’ll need to try various methods and ranges to generate the best model. To fine-tune your model, develop a script in BigQuery or an opensource tool like Python.
BigQuery ML vs. Traditional Methods
One of the most useful features inside BigQuery ML is the ability to specify standardized features equal to true, which streamlines your input to be on a normalized scale. Traditionally, this process would be completed manually using opensource tools like Scikit-learn. Instead, BigQuery does it for you.
It’s also simple to pass several transformations using BigQuery ML. Plus, BigQuery remembers the transformations, so that when you call predictions next time, it’s going to do the same transformations again. As a result, you receive all the useful information you care about when evaluating your model. You’ll see a breakdown of how many iterations your model went through, your clusters and how the input features vary across each cluster.
Once you save your model, it’s ready to serve predictions for you in real-time. There’s no need to move data from BigQuery to another tool such as Jupyter Notebooks. If you need to, you also have the option to download your models from classification, linear regression, or logistic regression and use them willfully.
BigQuery, TensorFlow & Jupyter Notebooks
BigQuery’s automated approach is easy to use, but it may not provide the flexibility you need to finetune your model. If you require high levels of accuracy, AI platform – which is hosted Jupyter Notebooks – and TensorFlow can help.
How They Work Together
By way of an example, if you have a baseline model and you need to explore a neural network, a customized machine learning solution is best. Or this might be the best route if you have innumerable timestamps or dense data.
With BigQuery, you can read data using TensorFlow’s reader from the BigQuery storage API, then read it into Jupyter Notebooks. Once you submit your training job, you have a final model you can then upload to BigQuery. Via this method, you’ll be able to serve your model where your data resides rather than going back and forth between AI platform and BigQuery.
You also have the option to train your model in Jupyter Notebooks and then deploy it to an AI platform for predictions from remote devices.
Make Google BigQuery ML Work for Your Organization
BigQuery ML reduces the time it takes you to build models by eliminating the need to move your data. Let BigQuery ML help you make critical business decisions using your data. To learn more about BigQuery ML, give us a call at 612-430-6316 or send us a message.