https://leafw-blog-pic.oss-cn-hangzhou.aliyuncs.com/avatar.jpg

Optimize Algorithms

This week’s content focuses on optimization algorithms, which can significantly enhance and expedite the training of deep learning models. Let’s dive in!

1. Importance of Optimization Algorithms

Optimization algorithms are crucial in the fields of machine learning and deep learning, particularly when training deep neural networks. These algorithms are methods used to minimize (or maximize) functions, typically the loss function in deep learning, with the goal of finding the optimal parameters that minimize this function.

Foundations of Practical Deep Learning

In the journey of learning deep learning, we encounter extensive theoretical knowledge, including gradient descent, backpropagation, and loss functions. A true understanding and application of these theories allow us to solve practical problems with ease. This blog, drawing from Week 1 of Course 2 in Professor Andrew Ng’s Deep Learning Specialization, explores critical concepts and methods from a practical standpoint. Key topics include how to divide training, development, and test sets, understanding and managing bias and variance, when and how to use regularization, and properly setting up optimization problems.

Introduction to Deep Learning and Neural Networks

The explosive popularity of ChatGPT and the recent flurry of large-scale model developments have thrust the field of artificial intelligence into the spotlight. As a tech enthusiast keen on exploring various technologies, understanding the principles behind these advancements is a natural inclination. Starting with deep learning is a logical step in delving deeper into AI, especially since I’ve already studied Professor Andrew Ng’s Machine Learning course. Now, through his Deep Learning Specialization, I am furthering my knowledge in this domain. This article aims to demystify deep learning, drawing insights from the first course of the specialization.

Brief Discussion on Distributed Transactions

In this article, we will focus on some relevant knowledge points about distributed transactions. This is an indispensable technology for learning distributed systems. The most common case is the bank transfer problem. Account A transfers 100 yuan to account B. Then the balance of account A should decrease by 100, and account B should increase by 100. The two steps must both succeed to be considered successful. If only one succeeds, it should be rolled back. If A and B are not in the same environment or system, then this transaction is a distributed transaction. So in this case, how to ensure the correct execution of the transaction and what execution plans are there?

Analysis of Spring Source Code (Part 2) — How to resolve circular dependencies

In the last Spring source code analysis, we skipped part of the code on Spring to solve the circular dependency part of the code, in order to fill the hole, I here another article to discuss this issue.

First of all, explain what is circular dependency, in fact, very simple, is that there are two classes they are dependent on each other, as follows.

@Component
public class AService {

    @Autowired
    private BService bService;
}
@Component
public class BService {
  
    @Autowired
    private AService aService;
}

AService and BService are obviously both internally dependent on each other, singled out as if to see the common deadlock code in the multi-threaded, but obviously Spring solves this problem, otherwise we would not be able to use it properly.

Analysis of Spring Source Code (Part 1) — Starting from the Spring Bean Lifecycle

The lifecycle of Spring Beans is really one of the most frequently asked questions about Spring in interviews. I have been stumped by this question more than once. I was wrong to try to rote memorize the steps. On the surface it looks like just reciting a process, but actually many interesting knowledge points are involved in this process.

The following diagram is probably something many people have seen the same or similar: