go-restful的介绍中,有介绍过Kubernetes中核心的API是使用go-restful来构建的,我们知道除了核心API,Kubernetes APIServer还提供了两种扩展机制,分别为Aggregation和APIExtensions,这两者中的API对象,则是使用NonGoRestfulMux来构建的,之所以不继续使用go-restful,原因还没有细究,可能是因为一些兼容性问题,后续有可能抛弃go-restful,完全切到NonGoRestfulMux上来,但是看改造工作量还是比较大的。下面我们来看看这个NonGoRestfulMux究竟是个什么东西,其代码位于:apiserver/pkg/server/mux/pathrecorder.go

其结构体如下:

type PathRecorderMux struct {
	// name is used for logging so you can trace requests through
	name string

	lock            sync.Mutex
	notFoundHandler http.Handler
	pathToHandler   map[string]http.Handler
	prefixToHandler map[string]http.Handler

	// mux stores a pathHandler and is used to handle the actual serving.
	// Turns out, we want to accept trailing slashes, BUT we don't care about handling
	// everything under them.  This does exactly matches only unless its explicitly requested to
	// do something different
	mux atomic.Value

	// exposedPaths is the list of paths that should be shown at /
	exposedPaths []string

	// pathStacks holds the stacks of all registered paths.  This allows us to show a more helpful message
	// before the "http: multiple registrations for %s" panic.
	pathStacks map[string]string
}

// pathHandler is an http.Handler that will satisfy requests first by exact match, then by prefix,
// then by notFoundHandler
type pathHandler struct {
	// muxName is used for logging so you can trace requests through
	muxName string

	// pathToHandler is a map of exactly matching request to its handler
	pathToHandler map[string]http.Handler

	// this has to be sorted by most slashes then by length
	prefixHandlers []prefixHandler

	// notFoundHandler is the handler to use for satisfying requests with no other match
	notFoundHandler http.Handler
}

// prefixHandler holds the prefix it should match and the handler to use
type prefixHandler struct {
	// prefix is the prefix to test for a request match
	prefix string
	// handler is used to satisfy matching requests
	handler http.Handler
}

PathRecorderMux即为NonGoRestfulMux,它里面有个原子类型的属性mux atomic.Value是其中最重要的属性,atomic是golang中一个用来做同步操作的数据类型,可以在不加锁的情况下,实现原子操作,通过store和load方法,来对其值进行存取。它存储的数据即为下面的pathHandlerpathHandler中又包含三个重要属性:pathToHandler, prefixHandlers以及notFoundHandler,即具体path到Handler的映射,某一个prefix到Handler的映射,以及路径没有匹配时映射的Handler,通过原子操作对其属性进行更新,而PathRecorderMux中的pathToHandlerprefixToHandler则都是为了能够线程安全的更新mux中的pathHandler而存在的。

PathRecorderMux的构建方法如下:

func NewPathRecorderMux(name string) *PathRecorderMux {
	ret := &PathRecorderMux{
		name:            name,
		pathToHandler:   map[string]http.Handler{},
		prefixToHandler: map[string]http.Handler{},
		mux:             atomic.Value{},
		exposedPaths:    []string{},
		pathStacks:      map[string]string{},
	}

	ret.mux.Store(&pathHandler{notFoundHandler: http.NotFoundHandler()})
	return ret
}

通过Store()方法将一个初始化的pathHandler存储到mux中。然后通过Handle(), HandleFunc(), HandlePrefix()等方法向PathRecorderMux中注册Handler,以Handle()方法为例:

func (m *PathRecorderMux) Handle(path string, handler http.Handler) {
	m.lock.Lock()
	defer m.lock.Unlock()
	m.trackCallers(path)

	m.exposedPaths = append(m.exposedPaths, path)
	m.pathToHandler[path] = handler
	m.refreshMuxLocked()
}

func (m *PathRecorderMux) refreshMuxLocked() {
	newMux := &pathHandler{
		muxName:         m.name,
		pathToHandler:   map[string]http.Handler{},
		prefixHandlers:  []prefixHandler{},
		notFoundHandler: http.NotFoundHandler(),
	}
	if m.notFoundHandler != nil {
		newMux.notFoundHandler = m.notFoundHandler
	}
	for path, handler := range m.pathToHandler {
		newMux.pathToHandler[path] = handler
	}

	keys := sets.StringKeySet(m.prefixToHandler).List()
	sort.Sort(sort.Reverse(byPrefixPriority(keys)))
	for _, prefix := range keys {
		newMux.prefixHandlers = append(newMux.prefixHandlers, prefixHandler{
			prefix:  prefix,
			handler: m.prefixToHandler[prefix],
		})
	}

	m.mux.Store(newMux)
}

可见,其通过加锁的方式,首先向PathRecorderMux中的pathToHandler中添加path到Handler的映射,然后调用refreshMuxLocked()方法,构建了一个新的pathHandler,然后使用PathRecorderMux中的属性更新该pathHandler,然后再将其存储到mux中,通过这种方式完成了原子更新。这里可能有个疑问,既然atomic可以不加锁就实现原子操作,为什么这里还要加锁,其实,这里还有个golang的知识点,就是golang中的map是线程不安全的,所以在多线程环境下,操作map,一般都需要通过加锁,保证线程安全。那么问题又来了,既然加锁了,那为啥不直接使用map,还费这么大劲,搞个atomic,其实我感觉不是不可以,只是使用atomic会更方便些,因为当你要读取atomic中的数据时,就不需要再加锁了,保证原子性。

下面来看看其ServeHTTP()方法,看它费这么大劲儿,到底是如何工作的:

// ServeHTTP makes it an http.Handler
func (m *PathRecorderMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	m.mux.Load().(*pathHandler).ServeHTTP(w, r)
}

// ServeHTTP makes it an http.Handler
func (h *pathHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	if exactHandler, ok := h.pathToHandler[r.URL.Path]; ok {
		klog.V(5).Infof("%v: %q satisfied by exact match", h.muxName, r.URL.Path)
		exactHandler.ServeHTTP(w, r)
		return
	}

	for _, prefixHandler := range h.prefixHandlers {
		if strings.HasPrefix(r.URL.Path, prefixHandler.prefix) {
			klog.V(5).Infof("%v: %q satisfied by prefix %v", h.muxName, r.URL.Path, prefixHandler.prefix)
			prefixHandler.handler.ServeHTTP(w, r)
			return
		}
	}

	klog.V(5).Infof("%v: %q satisfied by NotFoundHandler", h.muxName, r.URL.Path)
	h.notFoundHandler.ServeHTTP(w, r)
}

看到PathRecorderMux中实现的ServeHTTP()方法,通过load()读出mux中的数据,然后进行类型转换,转换成pathHandler指针,然后调用pathHandler的ServeHTTP()方法,在pathHandler的ServeHTTP()方法中,则可以看到NonGoRestfulMux的真正作用了,即分三步走:

  1. 首先从pathToHandler中找请求的路径跟注册的path完全匹配的,如果有,则由该Handler进行处理
  2. 如果没有完全匹配的,则看是否跟注册的prefix匹配,如果有,则由该Handler进行处理
  3. 如果上面都没匹配到,则由notFoundHandler进行处理

以上,就是NonGoRestfulMux的核心功能了。