Neural Network ABC: What is a Weight?

The basic computational unit of a neural network is the neuron, which aggregates incoming signals, applies a weighted sum, and passes the result through an activation function.

Assume y=mx + c, then we see the w as m. W is the weight.

To learn what is a weight, we can try to learn from this simple example.

What is a Weighted Average?A weighted average is a type of average where each value contributes to the final result based on its importance or weight. Unlike a simple (arithmetic) average where every value has equal influence, in a weighted average, values with higher weights have more impact.Formula




Simple Example: School GradesSuppose your final grade in a class is calculated as follows:

• Homework: 20% of the grade → average score = 85
• Quizzes: 30% of the grade → average score = 90
• Final Exam: 50% of the grade → score = 78

Here, the weights are the percentages (0.20, 0.30, 0.50), and the values are the scores.Step-by-step calculation:

1. Multiply each score by its weight:
   • Homework: 85 × 0.20 = 17.0
   • Quizzes: 90 × 0.30 = 27.0
   • Final Exam: 78 × 0.50 = 39.0
2. Sum these products:
   • 17.0 + 27.0 + 39.0 = 83.0
3. Sum of weights = 0.20 + 0.30 + 0.50 = 1.00 (or 100%)
4. Final grade = 83.0 / 1.00 = 83.0

Your final grade is 83.

References

Johnson, Peter. Fundamentals of Machine Learning: An Introduction to Neural Networks (p. 67). Kindle Edition. 

https://statisticsbyjim.com/basics/weighted-average/