Send Tiktok Data to Google BigQuery Using Node.js

by tyler garrett | Dec 29, 2022 | Automation, Google, Solutions

Here is an explanation of the code for sending TikTok data to Google BigQuery using Node.js:

const { BigQuery } = require('@google-cloud/bigquery');

This line imports the BigQuery class from the @google-cloud/bigquery library. The BigQuery class provides a client for interacting with the Big Query API.

async function sendTikTokDataToBigQuery(data) {
  // Create a client for interacting with the BigQuery API
  const bigquery = new BigQuery();

This function defines the sendTikTokDataToBigQuery function, which takes an array of data as an argument. The function begins by creating a new BigQuery client object.

// The name for the new dataset
  const datasetName = 'tiktok_data';

  // The name for the new table
  const tableName = 'tiktok_table';

These lines define the names of the new dataset and table that will be created in Big Query.

// The schema for the new table
  const schema = [
    { name: 'id', type: 'INTEGER' },
    { name: 'username', type: 'STRING' },
    { name: 'description', type: 'STRING' },
    { name: 'likes', type: 'INTEGER' },
    { name: 'comments', type: 'INTEGER' }
  ];

This defines the schema for the new table as an array of objects, with each object representing a column in the table and specifying the name and data type of the column.

// Create a new dataset
  await bigquery.createDataset(datasetName);

This line creates a new dataset in Big Query using the createDataset method of the bigquery client and the datasetName variable.

// Create a new table in the dataset
  await bigquery.dataset(datasetName).createTable(tableName, { schema: schema });

This line creates a new table in the dataset using the createTable method of the bigquery.dataset object and the tableName and schema variables.

// Insert the data into the table
  await bigquery
    .dataset(datasetName)
    .table(tableName)
    .insert(data);

This line inserts the data into the table using the insert method of the bigquery.dataset.table object and the data argument.

console.log(`Successfully sent TikTok data to Big Query: ${datasetName}.${tableName}`);
}

This logs a message indicating that the data has been successfully sent to Big Query.

const data = [
  { id: 1, username: 'tiktokuser1', description: 'My first TikTok video', likes: 1000, comments: 50 },
  { id: 2, username: 'tiktokuser2', description: 'My second TikTok video', likes: 2000, comments: 100 },
  { id: 3, username: 'tiktokuser3', description: 'My third TikTok video', likes: 3000, comments: 150 }
];
sendTikTokDataToBigQuery(data);

This code defines an array of TikTok data objects and then calls the sendTikTokDataToBigQuery function with this array as an argument. This will send the TikTok data to BigQuery.

The complete code to send TikTok data to Google Big Query using Node.js:

const { BigQuery } = require('@google-cloud/bigquery');

async function sendTikTokDataToBigQuery(data) {
  // Create a client for interacting with the BigQuery API
  const bigquery = new BigQuery();

  // The name for the new dataset
  const datasetName = 'tiktok_data';

  // The name for the new table
  const tableName = 'tiktok_table';

  // The schema for the new table
  const schema = [
    { name: 'id', type: 'INTEGER' },
    { name: 'username', type: 'STRING' },
    { name: 'description', type: 'STRING' },
    { name: 'likes', type: 'INTEGER' },
    { name: 'comments', type: 'INTEGER' }
  ];

  // Create a new dataset
  await bigquery.createDataset(datasetName);

  // Create a new table in the dataset
  await bigquery.dataset(datasetName).createTable(tableName, { schema: schema });

  // Insert the data into the table
  await bigquery
    .dataset(datasetName)
    .table(tableName)
    .insert(data);

  console.log(`Successfully sent TikTok data to Big Query: ${datasetName}.${tableName}`);
}

// Example usage: send TikTok data to Big Query
const data = [
  { id: 1, username: 'tiktokuser1', description: 'My first TikTok video', likes: 1000, comments: 50 },
  { id: 2, username: 'tiktokuser2', description: 'My second TikTok video', likes: 2000, comments: 100 },
  { id: 3, username: 'tiktokuser3', description: 'My third TikTok video', likes: 3000, comments: 150 }
];
sendTikTokDataToBigQuery(data);

This code creates a new Big Query dataset and table, and then inserts the TikTok data into the table. The schema for the table is defined as an array of objects, with each object representing a column in the table and specifying the name and data type of the column.

You will need to have the Google Cloud Big Query Node.js client library installed, which you can do by running npm install @google-cloud/bigquery in your project directory.

You will also need to have the necessary credentials for authenticating with the Big Query API. You can set up a service account and download the JSON key file from the Google Cloud Console, and then set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of the JSON key file.