Atom Feed

myersguo's blog 2019-03-22T11:02:38+08:00 myersguo event hook 2019-03-21T00:00:00+08:00 /2019/03/21/event-hook <p>event hook 是<a href="https://github.com/faif/python-patterns/blob/master/patterns/behavioral/observer.py">观察者模式</a>的典型应用。<code class="highlighter-rouge">provide a callback for notification of events/changes to data</code></p> <h3 id="django">django</h3> <p>用信号标识接收的方式,connect 表示接入, send 进行发送新号,所有接入方都将进行处理。</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Signal(object): def __init__(self): self.receivers = [] def connect(self, receiver): self.receivers.append(receiver) def send(self, sender, **named): responses = [] for receiver in self.receivers: response = receiver(**named) responses.append((receiver, response)) return responses test_signal = Signal() def test(): print " go test" test_signal.connect(test) test_signal.send(sender="sender") </code></pre></div></div> <h3 id="flask">flask</h3> <p>flask 里的 signal 使用 <code class="highlighter-rouge">blinker</code> 的 <code class="highlighter-rouge">Signal</code></p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Signal(object): def __init__(self): self.receivers = {} def connect(self, receiver): self.receivers.setdefault(receiver_id, receiver_ref) def send(self): return [(receiver, receiver(sender, **kwargs)) for receiver in self.receivers_for(sender)] </code></pre></div></div> <p>和 django 非常相似。</p> <h3 id="other">other</h3> <p>locust</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class EventHook(object): def __init__(self): self._handlers = [] def __iadd__(self, handler): self._handlers.append(handler) return self def fire(self, reverse=False, **kwargs): if reverse: self._handlers.reverse() for handler in self._handlers: handler(**kwargs) request_success = EventHook() def on_success(): print "on_success" request_success += on_success request_success.fire() </code></pre></div></div> <p>和 <code class="highlighter-rouge">django</code>, <code class="highlighter-rouge">flask</code> 类似。</p> decorator 2019-03-20T00:00:00+08:00 /2019/03/20/python-decorator <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def my_decorator(f): def wrapper(): print "before" f() print "after" return wrapper def test(): print "test" my_test = my_decorator(test) my_test() print test.__name__ print my_test.__name__ </code></pre></div></div> <p>每个函数作为「注解」的参数,注解的调用返回一个 wrapper 函数。 这个时候,my_test 变成了 <code class="highlighter-rouge">wrapper</code></p> <h3 id="语法糖-syntactic-sugar">@语法糖 syntactic sugar</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@my_decorator def test2(): print "test2" #test2 = my_decorator(test2) test2() </code></pre></div></div> <p>函数参数:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def my_decorator(f): def wrapper(*args, **kwargs): print "before" f(*args, **kwargs) print "after" return wrapper def test(name): print "test", name @my_decorator def test2(name): print "test2", name my_test = my_decorator(test) my_test("xiaoming") test2("xiaoming2") </code></pre></div></div> <p>注解带参数:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def with_params_decorator(n=0): def my_decorator(f): @functools.wraps(f) def wrapper(*args, **kwargs): print n print "before" f(*args, **kwargs) print "after" return wrapper return my_decorator </code></pre></div></div> <p>多套了一层。。。</p> <p>为了解决, python 类型自省变动问题,使用 <code class="highlighter-rouge">@functools.wrapper</code>来解决。来看下这个注解:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__doc__') WRAPPER_UPDATES = ('__dict__',) def update_wrapper(wrapper, wrapped, assigned = WRAPPER_ASSIGNMENTS, updated = WRAPPER_UPDATES): """Update a wrapper function to look like the wrapped function wrapper is the function to be updated wrapped is the original function assigned is a tuple naming the attributes assigned directly from the wrapped function to the wrapper function (defaults to functools.WRAPPER_ASSIGNMENTS) updated is a tuple naming the attributes of the wrapper that are updated with the corresponding attribute from the wrapped function (defaults to functools.WRAPPER_UPDATES) """ for attr in assigned: setattr(wrapper, attr, getattr(wrapped, attr)) for attr in updated: getattr(wrapper, attr).update(getattr(wrapped, attr, {})) # Return the wrapper so this can be used as a decorator via partial() return wrapper def wraps(wrapped, assigned = WRAPPER_ASSIGNMENTS, updated = WRAPPER_UPDATES): """Decorator factory to apply update_wrapper() to a wrapper function Returns a decorator that invokes update_wrapper() with the decorated function as the wrapper argument and the arguments to wraps() as the remaining arguments. Default arguments are as for update_wrapper(). This is a convenience function to simplify applying partial() to update_wrapper(). """ return partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated) </code></pre></div></div> <p>wraps 这个返回的是 partial 更新了的 <code class="highlighter-rouge">update_wrapper</code>,默认值 wrapped=wrapperd,即:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def wraps(wrapped): def new_wraps(): return update_wrapper(wrapped=wrappwd) return new_wraps </code></pre></div></div> <p>被 functions.wraps 注解的函数,的 name, doc 等都不会变。</p> <h3 id="类注解">类注解</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class class_decorator(object): def __init__(self, f): functools.update_wrapper(self, func) self.f = f def __call__(self, *args, **kwargs): print "before" self.f(*args, **kwargs) print "after" my_test = class_decorator(test) my_test("class") </code></pre></div></div> <p>类注解满足两点:1) <strong>init</strong> 传入 func, 2) <strong>call</strong> 可执行</p> what is thread local 2019-03-17T00:00:00+08:00 /2019/03/17/thread-local <p>进程之间是不共享内存的。不存在数据争强的问题,但是线程的数据是同内存种的,多线程模式下,对同一变量进行修改,可能造成数据不一致的情况。有两种解决办法:使用局部变量或者thread local variable。<br /> 使用局部变量有一个问题是在多个函数共同使用要通过参数传递方式实现。</p> <p>线程争强例子:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import threading total_cnt = 0 def process_1(): global total_cnt for x in xrange(500): total_cnt += 1 if __name__ == "__main__": for x in xrange(2): t = threading.Thread(target=process_1) t.start() print total_cnt </code></pre></div></div> <p>执行100次上面的代码: <br /> <code class="highlighter-rouge">for((x=0;x&lt;100;x+=1));do python thread_local.py;done</code> 你会发现结果不是全部都是 1000。 因为多线程下global variable 不是原子操作。对同一个数据操作需要加锁。</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>l = threading.Lock() def process_1(): global total_cnt if l.acquire(): try: for x in xrange(500): total_cnt += 1 finally: l.release() </code></pre></div></div> <p>local thread 一般用于解决线程之间数据隔离的问题。假如有一个 global 的数据,如果只在当前线程下有效,就可以使用 threading.local。 它是怎么实现的呢?可以参考<a href="https://github.com/pallets/werkzeug/blob/master/src/werkzeug/local.py">werkzeug</a>的实现。它使用<code class="highlighter-rouge">__storage__</code>来保存变量,每个线程使用不同的dict 的 id (通过 get_ident) 区分, 从而保证了线程的安全。</p> thrift 2019-03-14T00:00:00+08:00 /2019/03/14/thrift <p>先走一遍官网的例子:</p> <p>用到的两个文件<a href="https://github.com/apache/thrift/blob/master/tutorial/tutorial.thrift">tutorial.thrift</a> and <a href="https://github.com/apache/thrift/blob/master/tutorial/shared.thrift">shared.thrift</a>.</p> <p><code class="highlighter-rouge"> ~/thrift/bin/thrift -r --gen py:utf8strings -out ./thrift_gen ./tutorial.thrift </code></p> <p>生成三个文件: constants,ttypes,Calculator.py &amp;&amp; SharedService.py</p> <p>其中:constants 定义的是 const 变量,thrift 即里面的const 会放到这里:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>const i32 INT32CONSTANT = 9853 const map&lt;string,string&gt; MAPCONSTANT = {'hello':'world', 'goodnight':'moon'} </code></pre></div></div> <p>ttypes 里面定义 enum 类型和 struct 类型:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Operation: ADD = 1 SUBTRACT = 2 MULTIPLY = 3 DIVIDE = 4 </code></pre></div></div> <p>struct 会定义好 read,write 的方法:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Work: def __init__(...): pass def read(): ... def write(): ... </code></pre></div></div> <p>最后就是 service file ,里面定义暴露的接口:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def getStruct(self, key): self.send_getStruct(key) return self.recv_getStruct() </code></pre></div></div> <p><code class="highlighter-rouge">TProcessor</code>,<code class="highlighter-rouge">TTransport</code>,<code class="highlighter-rouge">TBinaryProtocol</code>,<code class="highlighter-rouge">TProtocol</code>,<code class="highlighter-rouge">fastbinary</code></p> <p>TProtocol 协议, TTransport: socket; TProcessor 方法处理;</p> <p>TProtocol 定义通信的协议。thrift TProtocolBase 协议定义了协议的开始、结束方法。writeMessageBegin 定义消息头,writeStructBegin 定义消息方法,writeFieldBegin 定义数据类型。eg:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> def writeMessageBegin(self, name, type, seqid): assert self.state == CLEAR self.__writeUByte(self.PROTOCOL_ID) self.__writeUByte(self.VERSION | (type &lt;&lt; self.TYPE_SHIFT_AMOUNT)) self.__writeVarint(seqid) self.__writeString(name) self.state = VALUE_WRITE def writeMessageEnd(self): assert self.state == VALUE_WRITE self.state = CLEAR def writeStructBegin(self, name): assert self.state in (CLEAR, CONTAINER_WRITE, VALUE_WRITE), self.state self.__structs.append((self.state, self.__last_fid)) self.state = FIELD_WRITE self.__last_fid = 0 </code></pre></div></div> <p>结合 <a href="https://gitbox.apache.org/repos/asf?p=thrift.git;a=blob_plain;f=tutorial/py/PythonClient.py;hb=HEAD">client</a>和<a href="https://gitbox.apache.org/repos/asf?p=thrift.git;a=blob;f=tutorial/py/PythonServer.py;hb=HEAD">server</a>的代码我们理解一下:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Make socket, 创建 socket, transport from thrift.transport import TSocket transport = TSocket.TSocket('localhost', 9090) # Buffering is critical. Raw sockets are very slow, from thrift.transport import TTransport transport = TTransport.TBufferedTransport(transport) # Wrap in a protocol, 协议初始化 protocol = TBinaryProtocol.TBinaryProtocol(transport) # Create a client to use the protocol encoder, 客户端 with proto client = Calculator.Client(protocol) # Connect! transport.open() client.ping() print('ping()') </code></pre></div></div> <p>processor 是解析、处理类,processMap是所有方法的 map。 每个 process 的过程是:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(name, type, seqid) = iprot.readMessageBegin() args.read() iprot.readMessageEnd() initResp hanle it: send recv write flush </code></pre></div></div> <p>transport 的过程:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> def read(self, sz): ret = self.__rbuf.read(sz) if len(ret) != 0: return ret self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size))) return self.__rbuf.read(sz) </code></pre></div></div> <p>TSocket:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import socket def open(self): try: res0 = self._resolveAddr() for res in res0: self.handle = socket.socket(res[0], res[1]) self.handle.settimeout(self._timeout) try: self.handle.connect(res[4]) except socket.error, e: if res is not res0[-1]: continue else: raise e break except socket.error, e: if self._unix_socket: message = 'Could not connect to socket %s' % self._unix_socket else: message = 'Could not connect to %s:%d' % (self.host, self.port) raise TTransportException(type=TTransportException.NOT_OPEN, message=message) </code></pre></div></div> <p>画重点,<strong>transport</strong> 就是 <strong><a href="https://docs.python.org/2/library/socket.html">socket</a></strong> 的传输。</p> <p>Client:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>send: self._oprot.writeMessageBegin args init self._oprot.writeMessageEnd flush recv: (fname, mtype, rseqid) = iprot.readMessageBegin() result.read(iprot) iprot.readMessageEnd() </code></pre></div></div> <p>看下 server 的过程:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class TSimpleServer(TServer): """Simple single-threaded server that just pumps around one transport.""" def __init__(self, *args): TServer.__init__(self, *args) def serve(self): self.serverTransport.listen() while True: client = self.serverTransport.accept() itrans = self.inputTransportFactory.getTransport(client) otrans = self.outputTransportFactory.getTransport(client) iprot = self.inputProtocolFactory.getProtocol(itrans) oprot = self.outputProtocolFactory.getProtocol(otrans) try: while True: self.processor.process(iprot, oprot) except TTransport.TTransportException, tx: pass except Exception, x: logging.exception(x) itrans.close() otrans.close() </code></pre></div></div> <p>process 一直跑….</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import socket sk = socket.socket() sk.bind(("127.0.0.1", 9092)) sk.listen(5) while True: client, addr = sk.accept() while True: data = client.recv(1024) if not data: break print "get from client:%s" % data client.sendall(data) client.close() client.py import socket s = socket.socket() s.connect(("127.0.0.1", 9092)) s.sendall("i am client") print "get from server:%s" % s.recv(1024) s.close() </code></pre></div></div> <p>整个流程下来就是:</p> <p>client 端: 建立socket 连接, 按照 proto 从 socket 里读数据。server 端:建立 socket,listen 连接, process 请求。</p> gevent 2019-03-08T00:00:00+08:00 /2019/03/08/gevent <h3 id="greenlet">greenlet</h3> <h3 id="hub--loop"><a href="http://www.gevent.org/api/gevent.hub.html">hub &amp;&amp; loop</a></h3> <p><a href="http://www.gevent.org/api/gevent.hub.html#the-event-loop">event loop</a>, hub 是一个单例的 greenlet, 用来调度 greenlet,每个 greenlet 都有一个 parent greenlet, loop server 在 hub里循环。<code class="highlighter-rouge">hub = get_hub(); loop = hub.loop;</code></p> <p>未完待续..</p> <h3 id="patch">patch</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def patch_item(module, attr, newitem): NONE = object() olditem = getattr(module, attr, NONE) if olditem is not NONE: saved.setdefault(module.__name__, {}).setdefault(attr, olditem) setattr(module, attr, newitem) def patch_module(name, items=None): gevent_module = getattr(__import__('gevent.' + name), name) module_name = getattr(gevent_module, '__target__', name) module = __import__(module_name) if items is None: items = getattr(gevent_module, '__implements__', None) if items is None: raise AttributeError('%r does not have __implements__' % gevent_module) for attr in items: patch_item(module, attr, getattr(gevent_module, attr)) </code></pre></div></div> <p>以上是 gevent 的 patch module 的过程。现在简化一下,方便理解:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># 当前目录有: #f1.py: def test(): print "test in current module" #modules/f1.py def test(): print "test in modules" #patch.py def patch_module(name): my_module = getattr(__import__('modules.' + name), name)#modules.f1 module_name = getattr(my_module, '__target__', name)#f1 module = __import__(module_name) #module f1 in current items = dir(my_module) for attr in items: setattr(module, attr, getattr(my_module, attr)) patch_module("f1") import f1 f1.test() </code></pre></div></div> <blockquote> <p>using <strong>import</strong> can be that it returns the imported module and doesn’t add anything to the namespace; you can import with it without having then to delete the new name if you didn’t want that new name; using import somename will add somename to your namespace, but <strong>import</strong>(‘somename’) instead returns the imported module, which you can then ignore.</p> </blockquote> <h3 id="geventthread">gevent.thread</h3> <p>patch 了以下方法:</p> <p>get_ident: 更改为获取当前协程的地址 <code class="highlighter-rouge">id(greenlet.getcurrent())</code></p> <p>start_new_thread:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def start_new_thread(function, args=(), kwargs={}): greenlet = Greenlet.spawn(function, *args, **kwargs) return get_ident(greenlet) </code></pre></div></div> 个人 OKR 实践 2019-02-22T00:00:00+08:00 /2019/02/22/my-okr <p>从 2.10 号开始,我开始尝试一个自我管理办法:用双周 OKR 来管理自己的目标。OKR 这个词最初是在小米使用,当时感觉像是在写周报一样,管理层也没有把 OKR 推广出去。来到头条后,发现每个人都要写OKR,从最上级到最底层得员工,OKR 向上对齐,完成公司的指定目标。</p> <p>某一天,我看到「凡事皆项目」,就知道我应该把自己也规划一下了。,然后写了以下 OKR:</p> <h4 id="210-224-project-g-build">2.10-2.24 project G build</h4> <ul> <li>读书与写作:读书习惯的初步建立 0.5 分(mysql 的书完成了30%, 中间时间因为某些原因断了) <ul> <li>上班路上养成阅读习惯</li> <li>完成 mysql 技术内部的阅读、redis 实践阅读1章</li> <li>每天下班路上完成3篇 micro notes</li> <li>完成2篇文章:一篇读书笔记、一篇思考</li> </ul> </li> <li>跑步:跑步习惯初步养成 0.8分 每周五坚持跑步了1小时,早起没有养成 <ul> <li>完成一次 10 公里</li> <li>每天6点30 准时起床</li> <li>每周一次跑步</li> </ul> </li> <li>工作:高效、高质量完成工作 0.5 分 ticket system 完成,mysql search 没有开展 <ul> <li>ticket system 完成提测、单元测试覆盖率 80% 以上</li> <li>初步完成 mysql 检索调研,完成一次笔记分享</li> </ul> </li> </ul> <p>事实证明, 规划自己的 OKR 是有效的。可以约束自己的行动,做到心中有数。</p> <p>现在互联网寒冬来了,而且,随着一个人的核心价值随着时间的推移而下降,不能通过规划来成长,就会逐渐被淘汰。</p> wordpress test 2019-01-06T00:00:00+08:00 /2019/01/06/dev-test <p>代码必须经过严格的测试才能够「放心」。我们通过 wordpress 的<a href="https://make.wordpress.org/core/handbook/testing/">test</a>来了解一下,如何测试一个 web server.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone git://develop.git.wordpress.org/ </code></pre></div></div> <p>wordpress 的测试包括但不限于<a href="https://make.wordpress.org/core/handbook/testing/beta/">beta testing</a>, <a href="https://make.wordpress.org/core/handbook/testing/user-testing/">user testing</a>, <a href="https://make.wordpress.org/core/handbook/testing/automated-testing/">automated testing, 自动化测试</a></p> <h3 id="自动化测试">自动化测试</h3> <p>自动化测试不需要手工接入,一般通过命令行运行测试集。</p> <p>wordpress 使用<a href="https://make.wordpress.org/core/handbook/testing/automated-testing/qunit/">qtest</a> 来测试 javascript。使用 phpunit 测试 php代码。</p> <p>自动化测试是一段验证 WP 函数的代码。test case 尽可能的小和独立。理想情况下,自动化测试case应该是单元测试,在没有外界依赖的隔离环境中可以验证。<strong>实际上</strong>,WordPress的架构和测试集,让写<code class="highlighter-rouge">pure</code>单测变得不可能。因此,这里的<strong>单测</strong>紧紧指自动化测试。</p> <h4 id="测试的架构">测试的架构</h4> <p><strong>任何测试</strong>最重要的部分就是<strong>断言</strong>(预期和结果)。许多测试都需要数据存储的支持。 WP 通常把数据存储在 DB 中。每一个独立的测试都从空的 wp 安装开始,因此每个测试都要为测试创造它自己的数据。这些数据被称为<strong>fixtures</strong>(通过 工厂函数产生)</p> <p>来感受一下:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&lt;?php class WP_UnitTest_Factory { public $post; public $attachment; public $comment; public $user; function __construct() { $this-&gt;post = new WP_UnitTest_Factory_For_Post( $this ); $this-&gt;attachment = new WP_UnitTest_Factory_For_Attachment( $this ); $this-&gt;user = new WP_UnitTest_Factory_For_User( $this ); } } </code></pre></div></div> <p>当一个 testcase 测试同一函数不通面时,组织 test function 到一个类。它能 share 同一个 setup。每一个 class 有一个 class file,</p> <p>每个用例测试完毕后,都要将数据<a href="https://github.com/WordPress/wordpress-develop/blob/master/tests/phpunit/includes/testcase.php#L85">清理干净</a></p> <p>看一下例子:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> </code></pre></div></div> auto develop 2019-01-03T00:00:00+08:00 /2019/01/03/auto-develop <p>我们通过 <a href="https://github.com/gin-gonic/gin/blob/master/.travis.yml">gin</a> 这个项目来看开源项目的开发与构建。</p> <h3 id="auto-build">auto build</h3> <p>登录到 <a href="https://docs.travis-ci.com/user/tutorial/">travis</a> 打开 auto build, 在项目根路径下添加 <a href="https://github.com/gin-gonic/gin/blob/master/.travis.yml">.travis.yml</a>, 然后等着每次 <a href="https://travis-ci.org/gin-gonic/gin">build</a>即可。</p> <p>从<a href="https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html">这篇文章</a> 和<a href="https://github.com/travis-ci/travis-ci">这里</a>我们可以粗略看出, travis 的基本组成:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Once a build request has been: accepted by Listener, approved and configured by Gatekeeper, and its jobs have been scheduled by Scheduler, they will be picked by a Worker which will execute the build Bash script as generated by the build script compiler). </code></pre></div></div> <p><a href="https://github.com/travis-ci/travis-web">travis-web</a> travis web client <br /> <a href="https://github.com/travis-ci/travis-api">api</a> travis 的核心逻辑层,提供运行服务的 api <br /> <a href="https://github.com/travis-ci/travis-build">travis-build</a>将travis.yml 转换成 build.sh,提供给 worker 执行 <br /> <a href="https://github.com/travis-ci/travis-hub">hub</a> 从其他 app 接收消息,然后通知其他app处理。比如,它通知task 模块, build 开始、完成事件。 <br /> <a href="https://github.com/travis-ci/travis-listener">travis-listener</a>接收 github 等的消息,然后发送到 mq<br /> <a href="https://github.com/travis-ci/travis-logs">travis-logs</a>Travis Logs processes log updates which are streamed from Travis Worker instances via RabbitMQ. The log parts are streamed via Pusher to the web client (Travis Web) and added to the database. <br /> <a href="https://github.com/travis-ci/travis-tasks">travis-tasks</a>接收任务的消息并发送通知 <br /> <a href="https://github.com/travis-ci/job-board">job-board</a> API 接口(代替异步的 mq) 来将构建任务传给 worker。<br /> <a href="https://github.com/travis-ci/travis-scheduler">schedule</a>接收触发请求(eg 来自github),创建一个构建任务(build record), <a href="https://github.com/travis-ci/worker">worker</a> 负责执行 ci job。</p> <h3 id="代码覆盖率">代码覆盖率</h3> <p>执行 make test 之后会在 travis config 中配置上传到<a href="https://codecov.io">https://codecov.io</a>, 其实就是执行的<a href="https://codecov.io/bash">这个脚本</a> <br /> <a href="https://github.com/codecov/example-go">参照</a>就可以看到<a href="https://codecov.io/gh/gin-gonic/gin">详细数据</a>了</p> <h3 id="docs">docs</h3> <p><a href="https://godoc.org/github.com/gin-gonic/gin">go doc</a> will auto generate</p> <h3 id="参考资料">参考资料</h3> <p><a href="https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html">https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html</a></p> 2018 2018-12-30T00:00:00+08:00 /2018/12/30/2018 <p>回顾<a href="http://myersguo.cn/2018/01/16/2017-2018.html">2017年</a>立下的 flag:</p> <p>跑步。跑一次10公里+;体重减20斤; <br /> 旅游。周末周边游若干次,一次远途的家庭旅行;<br /> 控制。减少手机使用量(走路、地铁上不看手机); 说话不要太「刺耳」;放慢去看世界; <br /> 读书。多看 &amp; 买的书都看完。 <br /> 家庭。对孩子更加耐心,和家人、孩子认真的相处;<br /> 其他。坚持写作。炒股盈利。开车技术大幅提升。</p> <p>太惭愧,只完成一项:跑步十公里。体重仍然没有减少。周边游依然搁置。路上仍然习惯看手机,说话更加刺耳,控制不住自己的情绪。读书依然是「静静地躺在书包里」的情况,写作没有坚持。炒股巨亏20%。开车技巧不但没有提升,信心大大降低。不过,还是有一些流水可以晒一下给自己的:</p> <p>沫沫已经上幼儿园了。刚开始在一家价钱比较便宜的幼儿园,一个班级有2/30号人,每天背着比自己都大的书包上学。后来换到了一个双语幼儿园,班里10个人左右,感觉她每天都比较开心,好像对现在的幼儿园比较满意,学会了自己整理玩具,礼貌。但吃饭还是挑食,大便还是比较干。睡觉还是要抱着她的「姐姐」,吃着手睡。</p> <p>近几个月沫沫妈妈总是出差,几乎没有完整的一个月是在家里的。她的事业黄金期是不是来的完了一些?</p> <p>我的工作在2018年感觉发生了一点变动。年初还在一种「纠结」的境况中,每天会纠结这一些事的「意义」,后来由于某些事情,终于让我抛弃了自己的「纠结」,我离开了干了4年的测开行业,正式转到开发。像是一只放飞了的小鸟,翱翔在自由的天空中,遇到的事情都那么清新,还记得我去旁边的XX办公地对需求,在马路边「抽风」拍照,仿佛回到了「年轻」的时候,哦,这才是我想要的 feel ? 不过,偶尔会想着,以后怎么向上爬的问题。打怪升级好像是我目前不得不关心的了?</p> <p>2019年,我的目标是把2018年的目标实现:</p> <p>跑步。2019年的目标是一次半马(21公里) <br /> 家庭与朋友。对家人耐心,对朋友真诚。珍惜和孩子的相处时光。 <br /> 读书与写作: 坚持每日写作,每周一本书 控制自己:情绪保持稳定,不要对人太刻薄,多关系他人;戒掉手机。 <br /> 旅游。必须来一次远途家庭旅游了。 <br /> 开车技巧大福提升,股票理财平稳进行</p> <p>2018 已经过去,2019, 新年快乐。</p> algorithm 2018-08-13T00:00:00+08:00 /2018/08/13/algorithm <h3 id="用两个堆栈模拟队列的功能-pushpopcount">用两个堆栈模拟队列的功能: push\pop\count</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def stack_push(a, b, item): """ 入队操作,判断 a 是否为空,为空则将b中数据压入a,不为空,则直接入栈 """ if len(a) &lt;= 0: while len(b) &gt; 0: a.append(b.pop()) a.append(item) def stack_pop(a, b): """ 出队操作:判断 b 是否为空,不为空直接弹出;为空则将a压入b中 """ if len(b) &gt;0: return b.pop() while len(a) &gt; 1: b.append(a.pop()) if len(a) &gt; 0: return a.pop() def stack_count(a, b): return len(a) if len(b) &lt;=0 else len(b) a, b = [], [] if __name__ == "__main__": stack_pop(a, b) stack_push(a, b, 1) stack_push(a, b, 3) stack_push(a, b, 5) assert stack_count(a, b) == 3 assert stack_pop(a, b) == 1 assert stack_pop(a, b) == 3 assert stack_count(a, b) == 1 stack_push(a, b, 4) assert stack_count(a, b) == 2 assert stack_pop(a, b) == 5 assert stack_pop(a, b) == 4 </code></pre></div></div> <h3 id="二叉树最长路径">二叉树最长路径</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># 二叉树节点 class Node(object): def __init__(self, elem=None, lChild=None, rChild=None): self.elem = elem self.lChild = lChild self.rChild = rChild self.parent = None def __str__(self): if self.elem: return str(self.elem) # 二叉树最长路径 def find_tree_max_path(root): """ 层次遍历二叉树,对每个节点进行标记,标记其父节点 """ treeList = [] treeList.append(root) _tmp, index = treeList[0], 0 while True: if _tmp.lChild is not None: treeList.append(_tmp.lChild) print 'add new item', _tmp.lChild _tmp.lChild.parent = _tmp if _tmp.rChild is not None: treeList.append(_tmp.rChild) print 'add new item', _tmp.rChild _tmp.rChild.parent = _tmp index +=1 if index &gt;= len(treeList): break _tmp = treeList[index] # 输出最长路径 item = treeList[len(treeList)-1] while item: print item item = item.parent if __name__ == "__main__": a, b, c, d, e, f, g = Node(1), Node(2), Node(3),Node(4),Node(5),Node(6),Node(7) a.lChild, a.rChild = b, c b.lChild, b.rChild = d, e e.lChild, e.rChild = f, g find_tree_max_path(a) </code></pre></div></div> <h3 id="android-解锁密码">android 解锁密码</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>from itertools import permutations, chain impossible = {'91': '5', '13': '2', '17': '4', '46': '5', '37': '5', '31': '2', '28': '5', '39': '6', '19': '5', '64': '5', '97': '8', '71': '4', '82': '5', '73': '5', '93': '6', '79': '8'} # 生成所有的组合类型 allIterate = chain(*(permutations('123456789', i) for i in range(4, 10))) count = 0 for item in allIterate: stri = ''.join(item) for k, v in impossible.items(): if k in stri and v not in stri[:stri.find(k)]: break else: count += 1 </code></pre></div></div> <h3 id="出现此处大于-m-的-数组元素">出现此处大于 m 的 数组元素</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>a = [1, 1, 3, 1, 2, 3, 1, 2, 3, 4, 5, 4, 5, 4, 1] def findArr(a, m=3): result = [] hashSet = {} for item in a: if hashSet.get(item): hashSet[item] +=1 else: hashSet[item] = 1 for item in hashSet: if hashSet[item] &gt; m: result.append(item) return result print findArr(a, 4) </code></pre></div></div> <p>假如我们期望 hashSet 的空间尽量小,则:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def findArr2(a, m=3): result, hashSet = [], {} lenA = len(a) lenH = lenA / m for item in a: if hashSet.get(item) is not None: if hashSet[item] &lt;=0: del hashSet[item] continue if len(hashSet) &gt; lenH: for _item in hashSet: hashSet[_item] -=1 hashSet[item] += 1 else: hashSet[item] = 1 for item in hashSet: if hashSet[item] &gt; m: result.append(item) return result </code></pre></div></div> <h3 id="有两个字典分别存有-100-条数据和-10000-条数据如果用一个不存在的-key-去查找数据在哪个字典中速度更快">有两个字典,分别存有 100 条数据和 10000 条数据,如果用一个不存在的 key 去查找数据,在哪个字典中速度更快?</h3> <p>hash 表的满的程度用负载因子体现。负载因子越大,意味着哈希表越满,越容易导致冲突,性能也就越低。因此,一般来说,当负载因子大于某个常数(可能是 1,或者 0.75 等)时,哈希表将自动扩容。<a href="https://bestswifter.com/hashtable/">参考</a>。hash 扩容 rehash 过程一般2倍空间的申请,将原来的数据复制到新的空间。</p> <p>redis的扩容,详见:<a href="http://redisbook.com/preview/dict/incremental_rehashing.html">渐进式hash</a></p>