Sentiment Analysis, also known as Opinion Mining, is a field of study that deals with the computational treatment of opinions, sentiments, evaluations, and emotions expressed in text. It is used to determine the polarity of a text, whether it is positive, negative, or neutral, and to quantify the degree of sentiment expressed. Sentiment analysis has become a critical tool for businesses, researchers, and governments to gain insights into public opinion and customer feedback, and to monitor social media for brand reputation management.
The Natural Language Toolkit (NLTK) is a popular open-source library in Python that provides a comprehensive suite of tools for working with human language data. One of the most useful features of NLTK is its SentimentIntensityAnalyzer, a pre-trained model that can be used to perform sentiment analysis on text data.
The SentimentIntensityAnalyzer uses a lexicon-based approach, where each word in a sentence is looked up in a pre-defined sentiment lexicon and given a sentiment score. In the case of the SentimentIntensityAnalyzer in NLTK, the sentiment lexicon used is the VADER (Valence Aware Dictionary and sentiment Reasoner) lexicon, which contains a large list of words and their associated sentiment scores. The scores in the VADER lexicon range from -1 (very negative) to +1 (very positive).
Here is an example of sentiment analysis in Python using the Natural Language Toolkit (nltk) library:
import nltk
from nltk.sentiment import SentimentIntensityAnalyzer
def sentiment_analysis(sentence):
# Use the SentimentIntensityAnalyzer to compute the sentiment scores
sentiment = SentimentIntensityAnalyzer().polarity_scores(sentence)
# Categorize the sentiment as positive, negative, or neutral based on the compound score
if sentiment['compound'] >= 0.05:
sentiment_category = "Positive"
elif sentiment['compound'] <= -0.05:
sentiment_category = "Negative"
else:
sentiment_category = "Neutral"
return sentiment, sentiment_category
# Test the sentiment analysis on some example sentences
sentence = "I love this youtube video! You Rock."
sentiment, sentiment_category = sentiment_analysis(sentence)
print("Sentence:", sentence)
print("Compound score:", sentiment['compound'])
print("Sentiment:", sentiment_category)
sentence = "I hate this youtube video! You're Terrible."
sentiment, sentiment_category = sentiment_analysis(sentence)
print("Sentence:", sentence)
print("Compound score:", sentiment['compound'])
print("Sentiment:", sentiment_category)
sentence = "I feel so-so about your youtube videos."
sentiment, sentiment_category = sentiment_analysis(sentence)
print("Sentence:", sentence)
print("Compound score:", sentiment['compound'])
print("Sentiment:", sentiment_category)
sentence = "I feel so-so about your boring youtube videos."
sentiment, sentiment_category = sentiment_analysis(sentence)
print("Sentence:", sentence)
print("Compound score:", sentiment['compound'])
print("Sentiment:", sentiment_category)
This code uses the SentimentIntensityAnalyzer from nltk to compute the sentiment scores for each sentence. The polarity_scores method returns a dictionary containing four values, which represent the sentiment of the sentence: pos, neg, neu, and compound. The compound score is a composite score that summarizes the overall sentiment of the sentence, where scores close to 1 indicate a positive sentiment, scores close to -1 indicate a negative sentiment, and scores close to 0 indicate a neutral sentiment. In this example, we use the compound score to categorize the sentiment of each sentence as positive, negative, or neutral.
What other tools are within SentimentIntensityAnalyzer?
The polarity_scores method of the SentimentIntensityAnalyzer in the nltk library returns a dictionary containing the following four values:
pos: The positive sentiment score, ranging from 0 to 1, where higher values indicate a more positive sentiment.
neg: The negative sentiment score, ranging from 0 to 1, where higher values indicate a more negative sentiment.
neu: The neutral sentiment score, ranging from 0 to 1, where higher values indicate a more neutral sentiment.
compound: The composite sentiment score, which summarizes the overall sentiment of the sentence. The score is a float between -1 and 1, where positive values indicate a positive sentiment, negative values indicate a negative sentiment, and values close to 0 indicate a neutral sentiment.
So in addition to the compound score, you can also use the pos, neg, and neu scores to further analyze the sentiment of a sentence. For example, you could use the pos and neg scores to see which sentiment is more strongly expressed in the sentence, or you could use the neu score to determine how much of the sentiment is neutral.
In sentiment analysis, how is compound aggregated?
The compound score in the nltk library’s SentimentIntensityAnalyzer is created using a weighted average of the scores for positive, negative, and neutral sentiment. The scores for each sentiment are generated using a lexicon-based approach, where each word in a sentence is looked up in a pre-defined sentiment lexicon and given a sentiment score.
In the case of the SentimentIntensityAnalyzer in the nltk library, the sentiment lexicon used is the VADER lexicon, which contains a large list of words and their associated sentiment scores. The scores in the VADER lexicon range from -1 (very negative) to +1 (very positive).
To compute the compound score for a sentence, the scores for each word in the sentence are first combined, taking into account the position of the word in the sentence and any intensifying or dampening words that may be present. This results in an intermediate score for the sentence. The final compound score is then computed as a weighted average of this intermediate score, the positive score, the negative score, and the neutral score.
The exact formula for the compound score is not publicly available, but it has been shown to be effective in accurately capturing the overall sentiment of a sentence.
Processing unstructured text can take various approaches. One way is to split paragraphs based on new lines and break sentences by focusing on spaces. However, this can lead to the need for sourcing your own scoring data to join with your unstructured data source, and that requires data warehousing services internally or externally. Finding, cleaning, and processing your word scoring data sources is a project that becomes a big part of this realm of solving. NLP can be considered a row-level relationship when using a relational database solution, but NLTK provides a Python alternative that eliminates the need for processing data differently than what a relational database would solve. An acid compliant database would prioritize establishing a relationship with “word scoring” tables, requiring scored data per word, which can be a time-consuming task. Instead, there are more precise and efficient methods available, such as those showcased in this blog post, which include resources like YouTube content, Python code, Code walkthroughs, and a cloud version of Jupyter notebook code that your digital marketing team can use to start solving problems immediately.
Both row level scoring per word and NLP are both power tools when trying to understand data. Data engineering services will open a new level of data solution development, and allow you to quickly harness different levels of capabilities with your internal and external data sources.
Natural Language Processing, or NLP for short, is a branch of artificial intelligence that deals with the interaction between computers and human languages. It is a field that has seen tremendous growth in recent years, with applications ranging from language translation to sentiment analysis, and even to building intelligent virtual assistants.
At its core, NLP is about teaching computers to understand and process human language. This is a challenging task, as human language is highly ambiguous and context-dependent. For example, the word “bass” can refer to a type of fish or a low-frequency sound, and the word “bat” can refer to an animal or a piece of sports equipment. Understanding the intended meaning in a given context requires a deep understanding of the language and the context in which it is used.
There are several key techniques that are used in NLP, including:
Tokenization: This is the process of breaking down a sentence or text into individual words or phrases. This is the first step in any NLP process, as it allows the computer to work with the individual elements of the text.
Part-of-speech tagging: This is the process of identifying the role of each word in a sentence, such as whether it is a noun, verb, adjective, etc. This helps the computer understand the grammar and structure of the sentence.
Named Entity Recognition: This is the process of identifying proper nouns and entities in a sentence such as people, places, and organizations. This can be used to extract structured information from unstructured text.
Sentiment Analysis: This is the process of determining the emotional tone of a piece of text. This can be used to understand how people feel about a particular topic or product.
Machine Translation: This is the process of converting text from one language to another. This can be used to translate documents, websites or even speech.
These are just a few examples of the many techniques used in NLP. The field is constantly evolving, with new techniques and algorithms being developed all the time. As the amount of text data available on the internet continues to grow, the importance of NLP will only increase. It is a fascinating field that has the potential to revolutionize how we interact with technology, and understanding the basics of NLP is essential for anyone working in technology or data science.
In conclusion, NLP is a rapidly growing field that deals with teaching computers to understand human languages. It encompasses a wide range of techniques and applications, from tokenization and part-of-speech tagging to sentiment analysis and machine translation. With the increasing amount of text data available, understanding the basics of NLP is essential for anyone working in technology or data science.
Here’s a polished rewrite of the piece with a more modern, consulting-savvy tone, built for blog or LinkedIn visibility—and a few links to dev3lop.com added in naturally to align with your offerings:
The Finance Industry’s Over-Reliance on Data: The Risks and Drawbacks
In today’s finance world, data is king. From algorithmic trading and credit risk modeling to customer segmentation and fraud detection, data-driven strategies have become the norm. But as the industry races to automate, quantify, and optimize, it risks leaning too far into the data pool—and drowning in unintended consequences.
At DEV3LOP, we help businesses strike a healthy balance between data automation and human insight. And in the finance space, that balance is more crucial than ever.
The Hidden Dangers of Biased Data
The finance industry’s love for data often masks a serious issue: bias. Much of the data being used is self-reported, outdated, or pulled from limited sources, creating blind spots in analysis. As a result, decision-making skews toward reinforcing past patterns rather than adapting to changing market realities or underserved demographics.
Without proper data audits or a strategy to ensure diverse inputs, financial institutions risk building tools and processes that serve the few—while alienating the many. This is why we recommend regular data quality assessments and architecture reviews as part of any analytics strategy.
Over-Automation Undermines Accountability
From robo-advisors to loan approval algorithms, automation is everywhere. But when you replace too much human oversight with machines, things go wrong—quietly and at scale.
Automated systems can deny loans, flag transactions, or misjudge creditworthiness based on flawed logic or outdated models. Worse, they often lack transparency. Customers and regulators are left in the dark, unable to understand why a decision was made. At DEV3, we advocate for hybrid systems—automated where it helps, but always layered with explain-ability and human review.
Innovation Takes a Backseat
Chasing yesterday’s KPIs can stall tomorrow’s progress. A hyper-focus on metrics and historical trends can trap teams in a feedback loop of incremental optimization instead of bold innovation.
Many financial organizations struggle to evolve beyond what’s measurable, ignoring qualitative insights, customer empathy, and market intuition. That’s where data consultants can play a key role—unlocking creative solutions that aren’t confined to dashboards and spreadsheets.
The Cost of Keeping Up
Data isn’t cheap. Between storage infrastructure, security protocols, machine learning tools, and full-time data science teams, even mid-size firms can find themselves buried in technical debt.
Smaller players in the financial sector face an uphill battle just to participate. And when data becomes a barrier instead of an advantage, it’s time to reassess your stack. At DEV3LOP, we’ve helped firms trim unnecessary tools, simplify pipelines, and find scalable alternatives to bloated platforms.
Privacy and Trust Are on the Line
As the industry continues to harvest and analyze personal financial behavior, trust hangs in the balance. Data breaches, misuse of personal information, and the creep factor of overly personalized insights can quickly damage brand reputation.
In a post-GDPR, privacy-first world, data ethics are no longer optional—they’re strategic. Transparency in how data is collected, processed, and applied is essential, and your systems need to be designed with accountability at their core.
The Bottom Line
Data is powerful, but it’s not a silver bullet. Over-reliance on it—especially without governance, context, or critical thinking—can introduce risk, erode trust, and stall innovation. The financial services industry needs to rethink the role of data: not as the driver, but as the co-pilot.
Need help building a smarter, more balanced approach to data strategy in finance? Let’s talk.
In recent years, artificial intelligence (AI) has become increasingly prevalent in the business world, and one AI technology that has gained a lot of attention is ChatGPT. ChatGPT, or Chat Generative Pre-trained Transformer, is a natural language processing (NLP) system developed by OpenAI that allows businesses to create chatbots that can understand and respond to human language in a conversational manner.
There are many potential uses for ChatGPT in small businesses, and one of the most obvious is customer service. ChatGPT chatbots can be used to handle simple customer inquiries, such as answering frequently asked questions or providing information about products and services. This can free up time for human customer service representatives to handle more complex issues, or allow businesses to provide 24/7 customer service without the need for staff to be available at all times.
Another potential use for ChatGPT in small businesses is for marketing and sales. ChatGPT chatbots can be used to engage with potential customers, providing information about products and services and helping to guide them through the sales process. This can be particularly useful for businesses with a high volume of leads, as it allows them to personalize the sales process for each individual customer.
In addition to customer service and sales, ChatGPT chatbots can also be used for a wide range of other applications in small businesses. For example, they can be used to automate simple tasks, such as scheduling appointments or sending reminders. They can also be used to gather feedback from customers or employees, allowing businesses to continuously improve their products and services.
Overall, ChatGPT has the potential to significantly impact small businesses by providing a cost-effective way to automate simple tasks, improve customer service, and generate leads. While there are certainly limitations to what ChatGPT chatbots can do, they can be a valuable tool for small businesses looking to streamline their operations and improve their bottom line.
20 ChatGPT use cases that could be huge opportunities for small businesses
“Ask a ChatGPT” – a chatbot service that allows small business owners to get quick answers to their most pressing questions, such as “How do I file my taxes?” or “What’s the best way to market my business on social media?”
“ChatGPT Concierge” – a chatbot that helps small business owners plan their day, by providing recommendations for local events, restaurants, and activities based on their interests and schedule.
“ChatGPT HR” – a chatbot that helps small business owners manage their human resources tasks, such as scheduling interviews, creating employee profiles, and tracking time off.
“ChatGPT Bookkeeper” – a chatbot that assists small business owners with their accounting and financial management tasks, such as creating invoices, tracking expenses, and generating financial reports.
“ChatGPT Personal Shopper” – a chatbot that helps small business owners find the perfect gifts for their clients, employees, and loved ones, based on their preferences and budget.
“ChatGPT Travel Agent” – a chatbot that helps small business owners plan their business trips, by providing recommendations for flights, hotels, and activities based on their destination and needs.
“ChatGPT Social Media Manager” – a chatbot that helps small business owners create and schedule social media posts, and provides tips and strategies for growing their online presence.
“ChatGPT Event Planner” – a chatbot that assists small business owners with the planning and execution of events, such as conferences, workshops, and networking events.
“ChatGPT Virtual Assistant” – a chatbot that helps small business owners with a wide range of tasks, such as scheduling appointments, managing emails, and creating to-do lists.
“ChatGPT Customer Service” – a chatbot that assists small business owners with handling customer inquiries, complaints, and feedback, and provides personalized recommendations for products and services.
“ChatGPT Marketing Assistant” – a chatbot that helps small business owners create and implement marketing campaigns, by providing ideas for content, targeting the right audience, and measuring the results.
“ChatGPT Project Manager” – a chatbot that assists small business owners with managing projects and tasks, by setting deadlines, assigning responsibilities, and tracking progress.
“ChatGPT Website Assistant” – a chatbot that helps small business owners create and maintain their website, by providing tips and resources for design, content, and search engine optimization (SEO).
“ChatGPT Virtual Receptionist” – a chatbot that handles incoming calls and inquiries for small businesses, by providing information about products and services, directing calls to the appropriate staff member, and scheduling appointments.
“ChatGPT Lead Generator” – a chatbot that helps small business owners identify and connect with potential customers, by gathering information about their interests and needs, and providing personalized recommendations.
“ChatGPT Personal Trainer” – a chatbot that helps small business owners create and follow personalized fitness plans, by providing exercises, nutrition advice, and progress tracking.
“ChatGPT Nutritionist” – a chatbot that assists small business owners with planning and tracking their meals, by providing personalized nutrition recommendations and recipes based on their goals and preferences.
“ChatGPT Gardener” – a chatbot that helps small business owners plan and care for their gardens, by providing tips and resources for selecting plants, soil preparation, and pest control.
“ChatGPT Lawyer” – a chatbot that assists small business owners with legal questions and tasks, such as creating contracts, filing trademarks, and understanding regulations.
“ChatGPT Travel Companion” – a chatbot that helps small business owners plan and enjoy their travels, by providing recommendations for activities, restaurants, and accommodations based on their destination and interests.
All of the use cases listed above are possible with ChatGPT, as long as the chatbot is programmed and trained to handle the specific tasks and responses required for each use case. For example, a ChatGPT chatbot could be programmed to provide answers to frequently asked questions, or to recommend local events and activities based on a user’s interests and schedule. Similarly, a ChatGPT chatbot could be trained to handle customer inquiries and complaints, or to assist with financial management tasks such as invoicing and expense tracking.
Example code for the ChatGPT solutions listed above;
<script> // Initialize ChatGPT with your GPT-3 API key const chatgpt = new ChatGPT('your-api-key');
// Set up the chatbot's message container and input field const messageContainer = document.getElementById('chatgpt-messages'); const inputField = document.getElementById('chatgpt-input');
// Define a function to handle the user's input const sendMessage = () => { // Get the user's message from the input field const message = inputField.value;
// Clear the input field inputField.value = '';
// Use ChatGPT to generate a response to the user's message chatgpt.send(message).then(response => { // Display the user's message and ChatGPT's response messageContainer.innerHTML += ` <div class="user-message">${message}</div> <div class="chatgpt-message">${response}</div> `;
// Scroll the message container to the bottom messageContainer.scrollTop = messageContainer.scrollHeight; }); };
// Add an event listener to the input field to send the message when the user hits Enter inputField.addEventListener('keydown', event => { if (event.key === 'Enter') { sendMessage(); } }); </script>
Data visualization is an essential tool for understanding and analyzing data, and there are countless options out there for creating visually appealing and effective graphics. But with so many choices, it can be tough to know which tool is the right one for you. Even with 10 years + experience in data visualization consulting experience we feel we are always hearing about a new software or open source library, however which data visualization tool do we think will be top 5 in 2023?
Here are our top 5 data visualization tools to consider:
Tableau: Tableau is a powerful and widely-used data visualization tool that allows users to create interactive dashboards and charts. It’s easy to use and offers a wide range of customization options.
Power BI: Although we started as a Tableau shop, we have migrated most engagements to Power BI because of it’s ability to out perform Tableau when creating data product with ease. However we will mention here Power BI requires some good Data Governance Practices (AKA creating a data warehouse) in place, or it’s hard to utilize Power BI.
Excel: Excel may not be the most sophisticated data visualization tool out there, but it’s a tried-and-true option that’s available on almost every computer. Plus, it’s easy to learn and use, making it a great choice for beginners.
D3.js: D3.js is a JavaScript library that allows users to create dynamic and interactive visualizations. It’s a great choice for those who are comfortable with coding and want to create custom visualizations.
Plotly: Plotly is a cloud-based data visualization tool that allows users to create a wide range of charts and graphs. It’s easy to use and offers a range of customization options.
Google Charts: Google Charts is a free and easy-to-use data visualization tool that’s available online. It offers a wide range of chart types and customization options, and is a great choice for those looking for a quick and simple solution.
In conclusion, data visualization is an incredibly powerful tool for understanding and analyzing data, and there are so many fantastic options out there for creating visually appealing and effective graphics. But with so many choices, it can be tough to know which tool is the right one for you. Our top 5 data visualization tools – Tableau, Excel, D3.js, Plotly, and Google Charts – are all excellent options that offer a range of customization options and cater to different skill levels. No matter which one you choose, the most important thing is to find a tool that works for you and your needs. So go out there and start visualizing your data like a pro! And remember, the most important thing is to have fun with it – because let’s face it, staring at spreadsheets all day isn’t exactly thrilling. Happy visualizing!
To send XML data to Google BigQuery using Node.js, you will need to use the BigQuery API.
Here’s an example of how you can do this:
First, you will need to set up a project in the Google Cloud Console and enable the BigQuery API.
Install the Google Cloud client library for Node.js by running the following command:
npm install @google-cloud/bigquery
Import the BigQuery client and authenticate your application by creating a JSON key file and setting the GOOGLE_APPLICATION_CREDENTIALS environment variable:
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
Next, you can create a dataset and table in BigQuery to hold the XML data. You can do this using the createDataset and createTable methods of the BigQuery client:
async function createDatasetAndTable() {
// Create a dataset
const dataset = bigquery.dataset('xml_dataset');
await dataset.create();
// Create a table in the dataset
const table = dataset.table('xml_table');
await table.create({
schema: 'xml:string',
});
}
To insert the XML data into the table, you can use the insert method of the Table object:
