Java進(jìn)階之高并發(fā)核心Selector詳解

更新時(shí)間：2021年05月13日 10:12:33 作者：cristianoxm

前幾篇文章介紹了Java高并發(fā)的一些基礎(chǔ)內(nèi)容,認(rèn)識(shí)了Channel,Buffer和Selector的基本用法，有了感性認(rèn)識(shí)之后，來看看Selector的底層是如何實(shí)現(xiàn)的。,需要的朋友可以參考下

一、Selector設(shè)計(jì)

筆者下載得是openjdk8的源碼, 畫出類圖

在這里插入圖片描述
比較清晰得看到，openjdk中Selector的實(shí)現(xiàn)是SelectorImpl,然后SelectorImpl又將職責(zé)委托給了具體的平臺(tái)，比如圖中框出的

linux2.6以后才有的EpollSelectorImpl
Windows平臺(tái)是WindowsSelectorImpl
MacOSX平臺(tái)是KQueueSelectorImpl

從名字也可以猜到，openjdk肯定在底層還是用epoll,kqueue，iocp這些技術(shù)來實(shí)現(xiàn)的I/O多路復(fù)用。

二、獲取Selector

眾所周知，Selector.open()可以得到一個(gè)Selector實(shí)例，怎么實(shí)現(xiàn)的呢？

// Selector.java
public static Selector open() throws IOException {
    // 首先找到provider,然后再打開Selector
    return SelectorProvider.provider().openSelector();
}

// java.nio.channels.spi.SelectorProvider
    public static SelectorProvider provider() {
    synchronized (lock) {
        if (provider != null)
            return provider;
        return AccessController.doPrivileged(
            new PrivilegedAction<SelectorProvider>() {
                public SelectorProvider run() {
                        if (loadProviderFromProperty())
                            return provider;
                        if (loadProviderAsService())
                            return provider;
                            // 這里就是打開Selector的真正方法
                        provider = sun.nio.ch.DefaultSelectorProvider.create();
                        return provider;
                    }
                });
    }
}

在openjdk中，每個(gè)操作系統(tǒng)都有一個(gè)sun.nio.ch.DefaultSelectorProvider實(shí)現(xiàn)，以solaris為例:

/**
 * Returns the default SelectorProvider.
 */
public static SelectorProvider create() {
    // 獲取OS名稱
    String osname = AccessController
        .doPrivileged(new GetPropertyAction("os.name"));
    // 根據(jù)名稱來創(chuàng)建不同的Selctor
    if (osname.equals("SunOS"))
        return createProvider("sun.nio.ch.DevPollSelectorProvider");
    if (osname.equals("Linux"))
        return createProvider("sun.nio.ch.EPollSelectorProvider");
    return new sun.nio.ch.PollSelectorProvider();
}

如果系統(tǒng)名稱是Linux的話，真正創(chuàng)建的是sun.nio.ch.EPollSelectorProvider。如果不是SunOS也不是Linux,就使用sun.nio.ch.PollSelectorProvider, 關(guān)于PollSelector有興趣的讀者自行了解下, 本文僅以實(shí)際常用的EpollSelector為例探討。

打開sun.nio.ch.EPollSelectorProvider查看openSelector方法

public AbstractSelector openSelector() throws IOException {
    return new EPollSelectorImpl(this);
}

很直觀，這樣我們?cè)贚inux平臺(tái)就得到了最終的Selector實(shí)現(xiàn):sun.nio.ch.EPollSelectorImpl

三、EPollSelector如何進(jìn)行select

epoll系統(tǒng)調(diào)用主要分為3個(gè)函數(shù)

epoll_create: 創(chuàng)建一個(gè)epollfd,并開辟epoll自己的內(nèi)核高速cache區(qū)，建立紅黑樹，分配好想要的size的內(nèi)存對(duì)象，建立一個(gè)list鏈表，用于存儲(chǔ)準(zhǔn)備就緒的事件。epoll_wait: 等待內(nèi)核返回IO事件epoll_ctl: 對(duì)新舊事件進(jìn)行新增修改或者刪除

3.1 Epoll fd的創(chuàng)建

EPollSelectorImpl的構(gòu)造器代碼如下:

EPollSelectorImpl(SelectorProvider sp) throws IOException {
    super(sp);
    // makePipe返回管道的2個(gè)文件描述符，編碼在一個(gè)long類型的變量中
    // 高32位代表讀 低32位代表寫
    // 使用pipe為了實(shí)現(xiàn)Selector的wakeup邏輯
    long pipeFds = IOUtil.makePipe(false);
    fd0 = (int) (pipeFds >>> 32);
    fd1 = (int) pipeFds;
    // 新建一個(gè)EPollArrayWrapper
    pollWrapper = new EPollArrayWrapper();
    pollWrapper.initInterrupt(fd0, fd1);
    fdToKey = new HashMap<>();
}

再看EPollArrayWrapper的初始化過程

EPollArrayWrapper() throws IOException {
    // creates the epoll file descriptor
    // 創(chuàng)建epoll fd
    epfd = epollCreate();

    // the epoll_event array passed to epoll_wait
    int allocationSize = NUM_EPOLLEVENTS * SIZE_EPOLLEVENT;
    pollArray = new AllocatedNativeObject(allocationSize, true);
    pollArrayAddress = pollArray.address();

    // eventHigh needed when using file descriptors > 64k
    if (OPEN_MAX > MAX_UPDATE_ARRAY_SIZE)
        eventsHigh = new HashMap<>();
}
private native int epollCreate();

在初始化過程中調(diào)用了epollCreate方法，這是個(gè)native方法。
打開

jdk/src/solaris/native/sun/nio/ch/EPollArrayWrapper.c

EPollArrayWrapper() throws IOException {
    // creates the epoll file descriptor
    // 創(chuàng)建epoll fd
    epfd = epollCreate();

    // the epoll_event array passed to epoll_wait
    int allocationSize = NUM_EPOLLEVENTS * SIZE_EPOLLEVENT;
    pollArray = new AllocatedNativeObject(allocationSize, true);
    pollArrayAddress = pollArray.address();

    // eventHigh needed when using file descriptors > 64k
    if (OPEN_MAX > MAX_UPDATE_ARRAY_SIZE)
        eventsHigh = new HashMap<>();
}
private native int epollCreate();

可以看到最后還是使用了操作系統(tǒng)的api: epoll_create函數(shù)

3.2 Epoll wait等待內(nèi)核IO事件

調(diào)用Selector.select(),最后會(huì)委托給各個(gè)實(shí)現(xiàn)的doSelect方法,限于篇幅不貼出太詳細(xì)的，這里看下EpollSelectorImpl的doSelect方法

protected int doSelect(long timeout) throws IOException {
    if (closed)
        throw new ClosedSelectorException();
    processDeregisterQueue();
    try {
        begin();
        // 真正的實(shí)現(xiàn)是這行
        pollWrapper.poll(timeout);
    } finally {
        end();
    }
    processDeregisterQueue();
    int numKeysUpdated = updateSelectedKeys();

    // 以下基本都是異常處理
    if (pollWrapper.interrupted()) {
        // Clear the wakeup pipe
        pollWrapper.putEventOps(pollWrapper.interruptedIndex(), 0);
        synchronized (interruptLock) {
            pollWrapper.clearInterrupted();
            IOUtil.drain(fd0);
            interruptTriggered = false;
        }
    }
    return numKeysUpdated;
}

然后我們?nèi)タ?code>pollWrapper.poll, 打開jdk/src/solaris/classes/sun/nio/ch/EPollArrayWrapper.java:

int poll(long timeout) throws IOException {
    updateRegistrations();
    // 這個(gè)epollWait是不是有點(diǎn)熟悉呢？
    updated = epollWait(pollArrayAddress, NUM_EPOLLEVENTS, timeout, epfd);
    for (int i=0; i<updated; i++) {
        if (getDescriptor(i) == incomingInterruptFD) {
            interruptedIndex = i;
            interrupted = true;
            break;
        }
    }
    return updated;
}

private native int epollWait(long pollAddress, int numfds, long timeout,
                             int epfd) throws IOException;

epollWait也是個(gè)native方法,打開c代碼一看:

JNIEXPORT jint JNICALL
Java_sun_nio_ch_EPollArrayWrapper_epollWait(JNIEnv *env, jobject this,
                                            jlong address, jint numfds,
                                            jlong timeout, jint epfd)
{
    struct epoll_event *events = jlong_to_ptr(address);
    int res;

    if (timeout <= 0) {           /* Indefinite or no wait */
        // 發(fā)起epoll_wait系統(tǒng)調(diào)用等待內(nèi)核事件
        RESTARTABLE(epoll_wait(epfd, events, numfds, timeout), res);
    } else {                      /* Bounded wait; bounded restarts */
        res = iepoll(epfd, events, numfds, timeout);
    }

    if (res < 0) {
        JNU_ThrowIOExceptionWithLastError(env, "epoll_wait failed");
    }
    return res;
}
=

可以看到，最后還是發(fā)起的epoll_wait系統(tǒng)調(diào)用.

3.3 epoll control以及openjdk對(duì)事件管理的封裝

JDK中對(duì)于注冊(cè)到Selector上的IO事件關(guān)系是使用SelectionKey來表示，代表了Channel感興趣的事件，如Read,Write,Connect,Accept.

調(diào)用Selector.register()時(shí)均會(huì)將事件存儲(chǔ)到EpollArrayWrapper的成員變量eventsLow和eventsHigh中

// events for file descriptors with registration changes pending, indexed
// by file descriptor and stored as bytes for efficiency reasons. For
// file descriptors higher than MAX_UPDATE_ARRAY_SIZE (unlimited case at
// least) then the update is stored in a map.
// 使用數(shù)組保存事件變更, 數(shù)組的最大長度是MAX_UPDATE_ARRAY_SIZE, 最大64*1024
private final byte[] eventsLow = new byte[MAX_UPDATE_ARRAY_SIZE];
// 超過數(shù)組長度的事件會(huì)緩存到這個(gè)map中，等待下次處理
private Map<Integer,Byte> eventsHigh;


/**
 * Sets the pending update events for the given file descriptor. This
 * method has no effect if the update events is already set to KILLED,
 * unless {@code force} is {@code true}.
 */
private void setUpdateEvents(int fd, byte events, boolean force) {
    // 判斷fd和數(shù)組長度
    if (fd < MAX_UPDATE_ARRAY_SIZE) {
        if ((eventsLow[fd] != KILLED) || force) {
            eventsLow[fd] = events;
        }
    } else {
        Integer key = Integer.valueOf(fd);
        if (!isEventsHighKilled(key) || force) {
            eventsHigh.put(key, Byte.valueOf(events));
        }
    }
}

上面看到EpollArrayWrapper.poll()的時(shí)候, 首先會(huì)調(diào)用updateRegistrations

/**
 * Returns the pending update events for the given file descriptor.
 */
private byte getUpdateEvents(int fd) {
    if (fd < MAX_UPDATE_ARRAY_SIZE) {
        return eventsLow[fd];
    } else {
        Byte result = eventsHigh.get(Integer.valueOf(fd));
        // result should never be null
        return result.byteValue();
    }
}

/**
 * Update the pending registrations.
 */
private void updateRegistrations() {
    synchronized (updateLock) {
        int j = 0;
        while (j < updateCount) {
            int fd = updateDescriptors[j];
            // 從保存的eventsLow和eventsHigh里取出事件
            short events = getUpdateEvents(fd);
            boolean isRegistered = registered.get(fd);
            int opcode = 0;

            if (events != KILLED) {
                // 判斷操作類型以傳給epoll_ctl
                // 沒有指定EPOLLET事件類型
                if (isRegistered) {
                    opcode = (events != 0) ? EPOLL_CTL_MOD : EPOLL_CTL_DEL;
                } else {
                    opcode = (events != 0) ? EPOLL_CTL_ADD : 0;
                }
                if (opcode != 0) {
                    // 熟悉的epoll_ctl
                    epollCtl(epfd, opcode, fd, events);
                    if (opcode == EPOLL_CTL_ADD) {
                        registered.set(fd);
                    } else if (opcode == EPOLL_CTL_DEL) {
                        registered.clear(fd);
                    }
                }
            }
            j++;
        }
        updateCount = 0;
    }
}
private native void epollCtl(int epfd, int opcode, int fd, int events);

在獲取到事件之后將操作委托給了epollCtl,這又是個(gè)native方法，打開相應(yīng)的c代碼一看:

JNIEXPORT void JNICALL
Java_sun_nio_ch_EPollArrayWrapper_epollCtl(JNIEnv *env, jobject this, jint epfd,
                                           jint opcode, jint fd, jint events)
{
    struct epoll_event event;
    int res;

    event.events = events;
    event.data.fd = fd;

    // 發(fā)起epoll_ctl調(diào)用來進(jìn)行IO事件的管理
    RESTARTABLE(epoll_ctl(epfd, (int)opcode, (int)fd, &event), res);

    /*
     * A channel may be registered with several Selectors. When each Selector
     * is polled a EPOLL_CTL_DEL op will be inserted into its pending update
     * list to remove the file descriptor from epoll. The "last" Selector will
     * close the file descriptor which automatically unregisters it from each
     * epoll descriptor. To avoid costly synchronization between Selectors we
     * allow pending updates to be processed, ignoring errors. The errors are
     * harmless as the last update for the file descriptor is guaranteed to
     * be EPOLL_CTL_DEL.
     */
    if (res < 0 && errno != EBADF && errno != ENOENT && errno != EPERM) {
        JNU_ThrowIOExceptionWithLastError(env, "epoll_ctl failed");
    }
}

原來還是我們的老朋友epoll_ctl.
有個(gè)小細(xì)節(jié)是jdk沒有指定ET(邊緣觸發(fā))還是LT(水平觸發(fā)),所以默認(rèn)會(huì)用LT:)

在AbstractSelectorImpl中有3個(gè)set保存事件

// Public views of the key sets
// 注冊(cè)的所有事件
private Set<SelectionKey> publicKeys;             // Immutable
// 內(nèi)核返回的IO事件封裝，表示哪些fd有數(shù)據(jù)可讀可寫
private Set<SelectionKey> publicSelectedKeys;     // Removal allowed, but not addition

// 取消的事件
private final Set<SelectionKey> cancelledKeys = new HashSet<SelectionKey>();

在EpollArrayWrapper.poll調(diào)用完成之后, 會(huì)調(diào)用updateSelectedKeys來更新上面的仨set

private int updateSelectedKeys() {
    int entries = pollWrapper.updated;
    int numKeysUpdated = 0;
    for (int i=0; i<entries; i++) {
        int nextFD = pollWrapper.getDescriptor(i);
        SelectionKeyImpl ski = fdToKey.get(Integer.valueOf(nextFD));
        // ski is null in the case of an interrupt
        if (ski != null) {
            int rOps = pollWrapper.getEventOps(i);
            if (selectedKeys.contains(ski)) {
                if (ski.channel.translateAndSetReadyOps(rOps, ski)) {
                    numKeysUpdated++;
                }
            } else {
                ski.channel.translateAndSetReadyOps(rOps, ski);
                if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
                    selectedKeys.add(ski);
                    numKeysUpdated++;
                }
            }
        }
    }
    return numKeysUpdated;

代碼很直白，拿出事件對(duì)set比對(duì)操作。

四、Selector類的相關(guān)方法

在這里插入圖片描述

重點(diǎn)注意四個(gè)方法

select(): 這是一個(gè)阻塞方法，調(diào)用該方法，會(huì)阻塞，直到返回一個(gè)有事件發(fā)生的selectionKey集合
selectNow() :非阻塞方法，獲取不到有事件發(fā)生的selectionKey集合，也會(huì)立即返回
select(long):阻塞方法，如果沒有獲取到有事件發(fā)生的selectionKey集合，阻塞指定的long時(shí)間
selectedKeys(): 返回全部selectionKey集合，不管是否有事件發(fā)生

在這里插入圖片描述

可以理解：selector一直在監(jiān)聽select()

五、Selector、SelectionKey、ServerScoketChannel、ScoketChannel的關(guān)系

在這里插入圖片描述

Server代碼：

public class NIOServer {
    public static void main(String[] args) throws Exception{

        //創(chuàng)建ServerSocketChannel -> ServerSocket

        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();

        //得到一個(gè)Selecor對(duì)象
        Selector selector = Selector.open();

        //綁定一個(gè)端口6666, 在服務(wù)器端監(jiān)聽
        serverSocketChannel.socket().bind(new InetSocketAddress(6666));
        //設(shè)置為非阻塞
        serverSocketChannel.configureBlocking(false);

        //把 serverSocketChannel 注冊(cè)到  selector 關(guān)心 事件為 OP_ACCEPT
        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

        System.out.println("注冊(cè)后的selectionkey 數(shù)量=" + selector.keys().size()); // 1



        //循環(huán)等待客戶端連接
        while (true) {

            //這里我們等待1秒，如果沒有事件發(fā)生, 返回
            if(selector.select(1000) == 0) { //沒有事件發(fā)生
                System.out.println("服務(wù)器等待了1秒，無連接");
                continue;
            }

            //如果返回的>0, 就獲取到相關(guān)的 selectionKey集合
            //1.如果返回的>0， 表示已經(jīng)獲取到關(guān)注的事件
            //2. selector.selectedKeys() 返回關(guān)注事件的集合
            //   通過 selectionKeys 反向獲取通道
            Set<SelectionKey> selectionKeys = selector.selectedKeys();
            System.out.println("selectionKeys 數(shù)量 = " + selectionKeys.size());

            //遍歷 Set<SelectionKey>, 使用迭代器遍歷
            Iterator<SelectionKey> keyIterator = selectionKeys.iterator();

            while (keyIterator.hasNext()) {
                //獲取到SelectionKey
                SelectionKey key = keyIterator.next();
                //根據(jù)key 對(duì)應(yīng)的通道發(fā)生的事件做相應(yīng)處理
                if(key.isAcceptable()) { //如果是 OP_ACCEPT, 有新的客戶端連接
                    //該該客戶端生成一個(gè) SocketChannel
                    SocketChannel socketChannel = serverSocketChannel.accept();
                    System.out.println("客戶端連接成功 生成了一個(gè) socketChannel " + socketChannel.hashCode());
                    //將  SocketChannel 設(shè)置為非阻塞
                    socketChannel.configureBlocking(false);
                    //將socketChannel 注冊(cè)到selector, 關(guān)注事件為 OP_READ， 同時(shí)給socketChannel
                    //關(guān)聯(lián)一個(gè)Buffer
                    socketChannel.register(selector, SelectionKey.OP_READ, ByteBuffer.allocate(1024));
                    System.out.println("客戶端連接后 ，注冊(cè)的selectionkey 數(shù)量=" + selector.keys().size()); //2,3,4..

                }
                if(key.isReadable()) {  //發(fā)生 OP_READ

                    //通過key 反向獲取到對(duì)應(yīng)channel
                    SocketChannel channel = (SocketChannel)key.channel();

                    //獲取到該channel關(guān)聯(lián)的buffer
                    ByteBuffer buffer = (ByteBuffer)key.attachment();
                    channel.read(buffer);
                    System.out.println("form 客戶端 " + new String(buffer.array()));

                }
                //手動(dòng)從集合中移動(dòng)當(dāng)前的selectionKey, 防止重復(fù)操作
                keyIterator.remove();

            }
        }
    }
}

Client代碼

public class NIOClient {
    public static void main(String[] args) throws Exception{

        //得到一個(gè)網(wǎng)絡(luò)通道
        SocketChannel socketChannel = SocketChannel.open();
        //設(shè)置非阻塞
        socketChannel.configureBlocking(false);
        //提供服務(wù)器端的ip 和 端口
        InetSocketAddress inetSocketAddress = new InetSocketAddress("127.0.0.1", 6666);
        //連接服務(wù)器
        if (!socketChannel.connect(inetSocketAddress)) {

            while (!socketChannel.finishConnect()) {
                System.out.println("因?yàn)檫B接需要時(shí)間，客戶端不會(huì)阻塞，可以做其它工作..");
            }
        }

        //...如果連接成功，就發(fā)送數(shù)據(jù)
        String str = "hello, 尚硅谷~";
        //Wraps a byte array into a buffer
        ByteBuffer buffer = ByteBuffer.wrap(str.getBytes());
        //發(fā)送數(shù)據(jù)，將 buffer 數(shù)據(jù)寫入 channel
        socketChannel.write(buffer);
        System.in.read();

    }
}

六、總結(jié)

在這里插入圖片描述
jdk中Selector是對(duì)操作系統(tǒng)的IO多路復(fù)用調(diào)用的一個(gè)封裝，在Linux中就是對(duì)epoll的封裝。epoll實(shí)質(zhì)上是將event loop交給了內(nèi)核，因?yàn)榫W(wǎng)絡(luò)數(shù)據(jù)都是首先到內(nèi)核的，直接內(nèi)核處理可以避免無謂的系統(tǒng)調(diào)用和數(shù)據(jù)拷貝, 性能是最好的。jdk中對(duì)IO事件的封裝是SelectionKey, 保存Channel關(guān)心的事件。