Atom Feed

myersguo's blog 2019-02-10T15:56:27+08:00 myersguo wordpress test 2019-01-06T00:00:00+08:00 /2019/01/06/dev-test <p>代码必须经过严格的测试才能够「放心」。我们通过 wordpress 的<a href="https://make.wordpress.org/core/handbook/testing/">test</a>来了解一下,如何测试一个 web server.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone git://develop.git.wordpress.org/ </code></pre></div></div> <p>wordpress 的测试包括但不限于<a href="https://make.wordpress.org/core/handbook/testing/beta/">beta testing</a>, <a href="https://make.wordpress.org/core/handbook/testing/user-testing/">user testing</a>, <a href="https://make.wordpress.org/core/handbook/testing/automated-testing/">automated testing, 自动化测试</a></p> <h3 id="自动化测试">自动化测试</h3> <p>自动化测试不需要手工接入,一般通过命令行运行测试集。</p> <p>wordpress 使用<a href="https://make.wordpress.org/core/handbook/testing/automated-testing/qunit/">qtest</a> 来测试 javascript。使用 phpunit 测试 php代码。</p> <p>自动化测试是一段验证 WP 函数的代码。test case 尽可能的小和独立。理想情况下,自动化测试case应该是单元测试,在没有外界依赖的隔离环境中可以验证。<strong>实际上</strong>,WordPress的架构和测试集,让写<code class="highlighter-rouge">pure</code>单测变得不可能。因此,这里的<strong>单测</strong>紧紧指自动化测试。</p> <h4 id="测试的架构">测试的架构</h4> <p><strong>任何测试</strong>最重要的部分就是<strong>断言</strong>(预期和结果)。许多测试都需要数据存储的支持。 WP 通常把数据存储在 DB 中。每一个独立的测试都从空的 wp 安装开始,因此每个测试都要为测试创造它自己的数据。这些数据被称为<strong>fixtures</strong>(通过 工厂函数产生)</p> <p>来感受一下:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&lt;?php class WP_UnitTest_Factory { public $post; public $attachment; public $comment; public $user; function __construct() { $this-&gt;post = new WP_UnitTest_Factory_For_Post( $this ); $this-&gt;attachment = new WP_UnitTest_Factory_For_Attachment( $this ); $this-&gt;user = new WP_UnitTest_Factory_For_User( $this ); } } </code></pre></div></div> <p>当一个 testcase 测试同一函数不通面时,组织 test function 到一个类。它能 share 同一个 setup。每一个 class 有一个 class file,</p> <p>每个用例测试完毕后,都要将数据<a href="https://github.com/WordPress/wordpress-develop/blob/master/tests/phpunit/includes/testcase.php#L85">清理干净</a></p> <p>看一下例子:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> </code></pre></div></div> auto develop 2019-01-03T00:00:00+08:00 /2019/01/03/auto-develop <p>我们通过 <a href="https://github.com/gin-gonic/gin/blob/master/.travis.yml">gin</a> 这个项目来看开源项目的开发与构建。</p> <h3 id="auto-build">auto build</h3> <p>登录到 <a href="https://docs.travis-ci.com/user/tutorial/">travis</a> 打开 auto build, 在项目根路径下添加 <a href="https://github.com/gin-gonic/gin/blob/master/.travis.yml">.travis.yml</a>, 然后等着每次 <a href="https://travis-ci.org/gin-gonic/gin">build</a>即可。</p> <p>从<a href="https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html">这篇文章</a> 和<a href="https://github.com/travis-ci/travis-ci">这里</a>我们可以粗略看出, travis 的基本组成:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Once a build request has been: accepted by Listener, approved and configured by Gatekeeper, and its jobs have been scheduled by Scheduler, they will be picked by a Worker which will execute the build Bash script as generated by the build script compiler). </code></pre></div></div> <p><a href="https://github.com/travis-ci/travis-web">travis-web</a> travis web client <br /> <a href="https://github.com/travis-ci/travis-api">api</a> travis 的核心逻辑层,提供运行服务的 api <br /> <a href="https://github.com/travis-ci/travis-build">travis-build</a>将travis.yml 转换成 build.sh,提供给 worker 执行 <br /> <a href="https://github.com/travis-ci/travis-hub">hub</a> 从其他 app 接收消息,然后通知其他app处理。比如,它通知task 模块, build 开始、完成事件。 <br /> <a href="https://github.com/travis-ci/travis-listener">travis-listener</a>接收 github 等的消息,然后发送到 mq<br /> <a href="https://github.com/travis-ci/travis-logs">travis-logs</a>Travis Logs processes log updates which are streamed from Travis Worker instances via RabbitMQ. The log parts are streamed via Pusher to the web client (Travis Web) and added to the database. <br /> <a href="https://github.com/travis-ci/travis-tasks">travis-tasks</a>接收任务的消息并发送通知 <br /> <a href="https://github.com/travis-ci/job-board">job-board</a> API 接口(代替异步的 mq) 来将构建任务传给 worker。<br /> <a href="https://github.com/travis-ci/travis-scheduler">schedule</a>接收触发请求(eg 来自github),创建一个构建任务(build record), <a href="https://github.com/travis-ci/worker">worker</a> 负责执行 ci job。</p> <h3 id="代码覆盖率">代码覆盖率</h3> <p>执行 make test 之后会在 travis config 中配置上传到<a href="https://codecov.io">https://codecov.io</a>, 其实就是执行的<a href="https://codecov.io/bash">这个脚本</a> <br /> <a href="https://github.com/codecov/example-go">参照</a>就可以看到<a href="https://codecov.io/gh/gin-gonic/gin">详细数据</a>了</p> <h3 id="docs">docs</h3> <p><a href="https://godoc.org/github.com/gin-gonic/gin">go doc</a> will auto generate</p> <h3 id="参考资料">参考资料</h3> <p><a href="https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html">https://www.paperplanes.de/2013/10/18/the-smallest-distributed-system.html</a></p> 2018 2018-12-30T00:00:00+08:00 /2018/12/30/2018 <p>回顾<a href="http://myersguo.cn/2018/01/16/2017-2018.html">2017年</a>立下的 flag:</p> <p>跑步。跑一次10公里+;体重减20斤; <br /> 旅游。周末周边游若干次,一次远途的家庭旅行;<br /> 控制。减少手机使用量(走路、地铁上不看手机); 说话不要太「刺耳」;放慢去看世界; <br /> 读书。多看 &amp; 买的书都看完。 <br /> 家庭。对孩子更加耐心,和家人、孩子认真的相处;<br /> 其他。坚持写作。炒股盈利。开车技术大幅提升。</p> <p>太惭愧,只完成一项:跑步十公里。体重仍然没有减少。周边游依然搁置。路上仍然习惯看手机,说话更加刺耳,控制不住自己的情绪。读书依然是「静静地躺在书包里」的情况,写作没有坚持。炒股巨亏20%。开车技巧不但没有提升,信心大大降低。不过,还是有一些流水可以晒一下给自己的:</p> <p>沫沫已经上幼儿园了。刚开始在一家价钱比较便宜的幼儿园,一个班级有2/30号人,每天背着比自己都大的书包上学。后来换到了一个双语幼儿园,班里10个人左右,感觉她每天都比较开心,好像对现在的幼儿园比较满意,学会了自己整理玩具,礼貌。但吃饭还是挑食,大便还是比较干。睡觉还是要抱着她的「姐姐」,吃着手睡。</p> <p>近几个月沫沫妈妈总是出差,几乎没有完整的一个月是在家里的。她的事业黄金期是不是来的完了一些?</p> <p>我的工作在2018年感觉发生了一点变动。年初还在一种「纠结」的境况中,每天会纠结这一些事的「意义」,后来由于某些事情,终于让我抛弃了自己的「纠结」,我离开了干了4年的测开行业,正式转到开发。像是一只放飞了的小鸟,翱翔在自由的天空中,遇到的事情都那么清新,还记得我去旁边的XX办公地对需求,在马路边「抽风」拍照,仿佛回到了「年轻」的时候,哦,这才是我想要的 feel ? 不过,偶尔会想着,以后怎么向上爬的问题。打怪升级好像是我目前不得不关心的了?</p> <p>2019年,我的目标是把2018年的目标实现:</p> <p>跑步。2019年的目标是一次半马(21公里) <br /> 家庭与朋友。对家人耐心,对朋友真诚。珍惜和孩子的相处时光。 <br /> 读书与写作: 坚持每日写作,每周一本书 控制自己:情绪保持稳定,不要对人太刻薄,多关系他人;戒掉手机。 <br /> 旅游。必须来一次远途家庭旅游了。 <br /> 开车技巧大福提升,股票理财平稳进行</p> <p>2018 已经过去,2019, 新年快乐。</p> algorithm 2018-08-13T00:00:00+08:00 /2018/08/13/algorithm <h3 id="用两个堆栈模拟队列的功能-pushpopcount">用两个堆栈模拟队列的功能: push\pop\count</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def stack_push(a, b, item): """ 入队操作,判断 a 是否为空,为空则将b中数据压入a,不为空,则直接入栈 """ if len(a) &lt;= 0: while len(b) &gt; 0: a.append(b.pop()) a.append(item) def stack_pop(a, b): """ 出队操作:判断 b 是否为空,不为空直接弹出;为空则将a压入b中 """ if len(b) &gt;0: return b.pop() while len(a) &gt; 1: b.append(a.pop()) if len(a) &gt; 0: return a.pop() def stack_count(a, b): return len(a) if len(b) &lt;=0 else len(b) a, b = [], [] if __name__ == "__main__": stack_pop(a, b) stack_push(a, b, 1) stack_push(a, b, 3) stack_push(a, b, 5) assert stack_count(a, b) == 3 assert stack_pop(a, b) == 1 assert stack_pop(a, b) == 3 assert stack_count(a, b) == 1 stack_push(a, b, 4) assert stack_count(a, b) == 2 assert stack_pop(a, b) == 5 assert stack_pop(a, b) == 4 </code></pre></div></div> <h3 id="二叉树最长路径">二叉树最长路径</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># 二叉树节点 class Node(object): def __init__(self, elem=None, lChild=None, rChild=None): self.elem = elem self.lChild = lChild self.rChild = rChild self.parent = None def __str__(self): if self.elem: return str(self.elem) # 二叉树最长路径 def find_tree_max_path(root): """ 层次遍历二叉树,对每个节点进行标记,标记其父节点 """ treeList = [] treeList.append(root) _tmp, index = treeList[0], 0 while True: if _tmp.lChild is not None: treeList.append(_tmp.lChild) print 'add new item', _tmp.lChild _tmp.lChild.parent = _tmp if _tmp.rChild is not None: treeList.append(_tmp.rChild) print 'add new item', _tmp.rChild _tmp.rChild.parent = _tmp index +=1 if index &gt;= len(treeList): break _tmp = treeList[index] # 输出最长路径 item = treeList[len(treeList)-1] while item: print item item = item.parent if __name__ == "__main__": a, b, c, d, e, f, g = Node(1), Node(2), Node(3),Node(4),Node(5),Node(6),Node(7) a.lChild, a.rChild = b, c b.lChild, b.rChild = d, e e.lChild, e.rChild = f, g find_tree_max_path(a) </code></pre></div></div> <h3 id="android-解锁密码">android 解锁密码</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>from itertools import permutations, chain impossible = {'91': '5', '13': '2', '17': '4', '46': '5', '37': '5', '31': '2', '28': '5', '39': '6', '19': '5', '64': '5', '97': '8', '71': '4', '82': '5', '73': '5', '93': '6', '79': '8'} # 生成所有的组合类型 allIterate = chain(*(permutations('123456789', i) for i in range(4, 10))) count = 0 for item in allIterate: stri = ''.join(item) for k, v in impossible.items(): if k in stri and v not in stri[:stri.find(k)]: break else: count += 1 </code></pre></div></div> <h3 id="出现此处大于-m-的-数组元素">出现此处大于 m 的 数组元素</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>a = [1, 1, 3, 1, 2, 3, 1, 2, 3, 4, 5, 4, 5, 4, 1] def findArr(a, m=3): result = [] hashSet = {} for item in a: if hashSet.get(item): hashSet[item] +=1 else: hashSet[item] = 1 for item in hashSet: if hashSet[item] &gt; m: result.append(item) return result print findArr(a, 4) </code></pre></div></div> <p>假如我们期望 hashSet 的空间尽量小,则:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def findArr2(a, m=3): result, hashSet = [], {} lenA = len(a) lenH = lenA / m for item in a: if hashSet.get(item) is not None: if hashSet[item] &lt;=0: del hashSet[item] continue if len(hashSet) &gt; lenH: for _item in hashSet: hashSet[_item] -=1 hashSet[item] += 1 else: hashSet[item] = 1 for item in hashSet: if hashSet[item] &gt; m: result.append(item) return result </code></pre></div></div> <h3 id="有两个字典分别存有-100-条数据和-10000-条数据如果用一个不存在的-key-去查找数据在哪个字典中速度更快">有两个字典,分别存有 100 条数据和 10000 条数据,如果用一个不存在的 key 去查找数据,在哪个字典中速度更快?</h3> <p>hash 表的满的程度用负载因子体现。负载因子越大,意味着哈希表越满,越容易导致冲突,性能也就越低。因此,一般来说,当负载因子大于某个常数(可能是 1,或者 0.75 等)时,哈希表将自动扩容。<a href="https://bestswifter.com/hashtable/">参考</a>。hash 扩容 rehash 过程一般2倍空间的申请,将原来的数据复制到新的空间。</p> <p>redis的扩容,详见:<a href="http://redisbook.com/preview/dict/incremental_rehashing.html">渐进式hash</a></p> consul 2018-07-26T00:00:00+08:00 /2018/07/26/consul <h3 id="概念">概念</h3> <p>nodes: 实例节点 <br /> services: 服务(一般是 http/rpc client)</p> <h3 id="install">install</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># download from :https://www.consul.io/downloads.html # start agent consul agent -dev # test consul members curl localhost:8500/v1/catalog/nodes # tag service echo '{"service": {"name": "web", "tags": ["rails"], "port": 80}}' \ | sudo tee /etc/consul.d/web.json </code></pre></div></div> <p>test dns:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dig @127.0.0.1 -p 8600 web.service.consul web.service.consul. 0 IN A 127.0.0.1 </code></pre></div></div> <h3 id="http-api">HTTP API</h3> <p>set key v</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl --request PUT --data bar http://localhost:8500/v1/kv/foo </code></pre></div></div> <p>get key v</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/kv/foo </code></pre></div></div> <p>delete key v</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl --request DELETE http://localhost:8500/v1/kv/foo </code></pre></div></div> <h4 id="agent">agent</h4> <p>agent members:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/agent/members </code></pre></div></div> <p>reload</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl --request PUT http://localhost:8500/v1/agent/reload </code></pre></div></div> <p>log monitor:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://localhost:8500/v1/agent/monitor?loglevel=WARN </code></pre></div></div> <h4 id="event">event</h4> <p>show</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/event/list </code></pre></div></div> <h4 id="catalog">catalog</h4> <p>list node:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/catalog/nodes </code></pre></div></div> <p>dc list:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/catalog/datacenters </code></pre></div></div> <p>services:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/catalog/services </code></pre></div></div> <p><em>List Nodes for Service</em>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://localhost:8500/v1/catalog/services/myersguo.github.com </code></pre></div></div> <p><em>List Service for node</em>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl http://10.8.163.168:2280/v1/catalog/node/n6-028-xxx </code></pre></div></div> flask 2018-07-16T00:00:00+08:00 /2018/07/16/about-flask <p>flask 的 uwsgi 是基于<a href="http://werkzeug.pocoo.org/">werkzeug</a>, django 是自己实现了一套 wsgi 协议。</p> <h3 id="路由解析">路由解析</h3> <p>flask 的路由由 werkzeug 的 routing 模块实现,我们来看 flask 的封装:</p> <p>入口 <code class="highlighter-rouge">wsgi_app</code> :</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ctx = self.request_context(environ) # return RequestContext(self, environ) ctx.push() error = None try: try: response = self.full_dispatch_request() except Exception as e: error = e response = self.make_response(self.handle_exception(e)) return response(environ, start_response) finally: if self.should_ignore_error(error): error = None ctx.auto_pop(error) </code></pre></div></div> <h4 id="dispatch_request">dispatch_request</h4> <p>`return self.view_functions<a href="**req.view_args">rule.endpoint</a><br /> 这里的 view_functions 是在添加规则时(add_url_rule)设置的, 路由 rule 与 对应的 view 关系 list。</p> <h4 id="url_adapter-url_map">url_adapter, url_map</h4> <p>werkzeug.routing:</p> <p>create_url_adapter:创建一个 url dapater<br /> url_adapter.match: 获取匹配的 url rule<br /> self.view_functions: <a href="**req.view_args">rule.endpoint</a>: 执行对应的 function</p> <p>app.add_url_rule(): 添加路由,相当于注解 app.route:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def index(): pass app.add_url_rule('/', 'index', index) 相当于 @app.route('/') def index(): pass </code></pre></div></div> <p>rule 包括三个部分: converter, arguments, variable<br /> converter(arguments):name&gt;<br /> 比如:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import os from werkzeug.wrappers import Request, Response from werkzeug.wsgi import SharedDataMiddleware from werkzeug.routing import Map, Rule from werkzeug.exceptions import HTTPException, NotFound from werkzeug.routing import BaseConverter class RegexConverter(BaseConverter): def __init__(self, url_map, *args): super(RegexConverter, self).__init__(url_map) self.regex = args[0] class Shortly(object): def __init__(self): self.url_map = Map([ Rule('/', endpoint='new_url'), Rule('/&lt;regex(".*"):others&gt;', endpoint='new_url') ], converters={'regex': RegexConverter}) def dispatch_request(self, request): return Response('Hello World!') def dispatch_request(self, request): adapter = self.url_map.bind_to_environ(request.environ) try: endpoint, values = adapter.match() return getattr(self, 'on_' + endpoint)(request, **values) except HTTPException, e: return e def on_new_url(self, request, others=None): return Response('Hello World!') def wsgi_app(self, environ, start_response): request = Request(environ) response = self.dispatch_request(request) return response(environ, start_response) def __call__(self, environ, start_response): return self. wsgi_app(environ, start_response) def create_app(redis_host='localhost', redis_port=6379, with_static=True): app = Shortly() if with_static: app.wsgi_app = SharedDataMiddleware(app.wsgi_app, { '/static': os.path.join(os.path.dirname(__file__), 'static') }) return app if __name__ == '__main__': from werkzeug.serving import run_simple app = create_app() run_simple('127.0.0.1', 5000, app, use_debugger=True, use_reloader=True) </code></pre></div></div> <h3 id="middleware">middleware</h3> <p>app.wsgi_app = MyMiddleware(app.wsgi_app)</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>flask 的中间件是通过一些阶段方法来实现(before, after, tear down) </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>before_request: app.before_request(f) after_request: app.after_request(f) teardown_request: app.teardown_request(f) </code></pre></div></div> <h3 id="ctxflaskctx-请求上下文">ctx(flask.ctx) 请求上下文</h3> <p><strong><em>RequestContext</em></strong><br /> The request context contains all request relevant information. It is created at the beginning of the request and pushed to the <code class="highlighter-rouge">_request_ctx_stack</code> and removed at the end of it. It will create the URL adapter and request object for the WSGI environment provided</p> <p><strong><em>AppContext</em></strong>:<br /> The application context binds an application object implicitly to the current thread or greenlet.The application context is also implicitly created if a request context is created but the application is not on top of the individual application contex</p> <h3 id="helloworld">hello,world</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># coding: utf-8 from flask import Flask from flask import render_template from flask import request app = Flask(__name__) @app.route('/') def hello_world(): name = request.args.get('name', 'world') return 'Hello, ' + name @app.route('/hello/&lt;name&gt;') def hello(name=None): return render_template('hello.html', name=name) </code></pre></div></div> <p>flask run –host=0.0.0.0 <br /> or uwsgi -s /tmp/uwsgi.sock –module myapp –callable app –http 0.0.0.0:5000</p> <p>换成 django style 写 hello,world:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># index.py # coding: utf-8 from flask import Flask def create_app(): app = Flask(__name__, static_url_path='/static', static_folder="/webroot") import urls index = 0 for pattern in urls.urlpatterns: rule, view_func = pattern index += 1 app.add_url_rule(rule, endpoint=str(index), view_func=view_func) return app app = create_app() #urls.py import views urlpatterns = [ ('/', views.hello), ] # views.py def hello(): return 'Hello, World!' </code></pre></div></div> <p>更换成 views 类的方式:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#urls.py import views urlpatterns = [ ('/', views.MyView.as_view("index")), ] #views.py from flask import views class MyView(views.View): methods = ['GET'] def dispatch_request(self): return 'Hello World' </code></pre></div></div> <p>我们来看下 <code class="highlighter-rouge">flask.views</code> 这个类做了什么:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@classmethod def as_view(cls, name, *class_args, **class_kwargs): def view(*args, **kwargs): self = view.view_class(*class_args, **class_kwargs) return self.dispatch_request(*args, **kwargs) if cls.decorators: view.__name__ = name view.__module__ = cls.__module__ for decorator in cls.decorators: view = decorator(view) view.view_class = cls view.__name__ = name view.__doc__ = cls.__doc__ view.__module__ = cls.__module__ view.methods = cls.methods view.provide_automatic_options = cls.provide_automatic_options return view </code></pre></div></div> <h3 id="参考资料">参考资料</h3> <p><a href="https://blog.tonyseek.com/post/the-context-mechanism-of-flask/">https://blog.tonyseek.com/post/the-context-mechanism-of-flask/</a> <a href="http://werkzeug-docs-cn.readthedocs.io/zh_CN/latest/tutorial.html">http://werkzeug-docs-cn.readthedocs.io/zh_CN/latest/tutorial.html</a> <a href="https://jachinlin.github.io/2017/09/12/Werkzeug-URL-Routing/">https://jachinlin.github.io/2017/09/12/Werkzeug-URL-Routing/</a></p> minicap 一瞥 2018-06-06T00:00:00+08:00 /2018/06/06/about-minicap <h4 id="本地环境">本地环境</h4> <p>android sdk, android ndk-build tools(可以直接通过 android sdk manager安装)</p> <p>设置环境变量:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#ANDROID export ANDROID_HOME='/Users/guoliangyong/Library/Android/sdk' export PATH=$PATH:$ANDROID_HOME/platform-tools:$ANDROID_HOME/build-tools/26.0.0:$ANDROID_HOME/ndk-bundle </code></pre></div></div> <p>下载源码:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/openstf/minicap.git cd minicap &amp;&amp; git submodule init &amp;&amp; git submodule update ./run.sh autosize </code></pre></div></div> <h4 id="源码一瞥">源码一瞥</h4> 长时间没有写 blog 了 2018-05-15T00:00:00+08:00 /2018/05/15/long-time-no-write <p>已经一个多月没有写 blog 了,这么长时间里,我还是很平静的过着每一天。</p> <p>房租涨了 1千,北京的房子真是无可救药了(房租已经均价8千多了, 每平米均价88)。可怜。。。</p> <p>钱能解决的事情都不是事情,但「我没有钱啊。。。」</p> <p>技术上还是进步不大。。。</p> <p>「40岁了,还在写代码,是一种什么样的体验」? <br /> 中年危机,但是「做自己喜欢且擅长的事情」不是挺好?关键是,如果你聊到你「不可能擅长」或者做到「最好」,你会怎么去做?</p> <p>我不是为了变成 top 1% 的人,而是享受这个过程。在享受的过程中,慢慢地,变成了 top 1% ?</p> <p><a href="https://en.wikipedia.org/wiki/Quality_of_life">Quality of life</a> 的提升?</p> Gin, Golang HTTP web framework 2018-04-18T00:00:00+08:00 /2018/04/18/golang-gin-http-web-framework <p>近期几个 api server 使用 <a href="https://github.com/gin-gonic/gin">gin</a> 框架来实现。这里记录下 gin 的基本使用。 <br /> 来看 gin 的<code class="highlighter-rouge">hello,world</code>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span> <span class="n">import</span> <span class="s2">"github.com/gin-gonic/gin"</span> <span class="n">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="n">r</span> <span class="p">:=</span> <span class="n">gin</span><span class="p">.</span><span class="n">Default</span><span class="p">()</span> <span class="n">r</span><span class="p">.</span><span class="n">GET</span><span class="p">(</span><span class="s2">"/ping"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span><span class="m">200</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span> <span class="s2">"message"</span><span class="p">:</span> <span class="s2">"pong"</span><span class="p">,</span> <span class="p">})</span> <span class="p">})</span> <span class="n">r</span><span class="p">.</span><span class="nf">Run</span><span class="p">()</span> <span class="p">//</span> <span class="n">listen</span> <span class="k">and</span> <span class="n">serve</span> <span class="n">on</span> <span class="m">0.0.0.0</span><span class="p">:</span><span class="m">8080</span> <span class="p">}</span> </code></pre></div></div> <p>ok, 如果我们使用原生的 <a href="https://golang.org/doc/articles/wiki/">http 服务</a>怎么写呢?</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span> <span class="n">import</span> <span class="p">(</span> <span class="s2">"net/http"</span> <span class="p">)</span> <span class="n">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="n">http</span><span class="p">.</span><span class="n">HandleFunc</span><span class="p">(</span><span class="s2">"/ping"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">w</span> <span class="n">http</span><span class="p">.</span><span class="n">ResponseWriter</span><span class="p">,</span> <span class="n">r</span> <span class="p">*</span><span class="n">http</span><span class="p">.</span><span class="n">Request</span><span class="p">)</span> <span class="p">{</span> <span class="n">w</span><span class="p">.</span><span class="n">Header</span><span class="p">().</span><span class="k">Set</span><span class="p">(</span><span class="s2">"Content-Type"</span><span class="p">,</span> <span class="s2">"application/json"</span><span class="p">)</span> <span class="n">w</span><span class="p">.</span><span class="n">Write</span><span class="p">([]</span><span class="n">byte</span><span class="p">(</span><span class="s2">"{</span><span class="se">\"</span><span class="s2">message</span><span class="se">\"</span><span class="s2">:</span><span class="se">\"</span><span class="s2">pong</span><span class="se">\"</span><span class="s2">}"</span><span class="p">))</span> <span class="p">})</span> <span class="n">http</span><span class="p">.</span><span class="n">ListenAndServe</span><span class="p">(</span><span class="s2">":9095"</span><span class="p">,</span> <span class="n">nil</span><span class="p">)</span> <span class="p">}</span> </code></pre></div></div> <p>看上去差不太多?再看 gin 提供的强大能力,把官方例子下载到本地: <br /> <code class="highlighter-rouge">curl https://raw.githubusercontent.com/gin-gonic/gin/master/examples/basic/main.go &gt; main.go</code></p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span> <span class="n">import</span> <span class="p">(</span> <span class="s2">"github.com/gin-gonic/gin"</span> <span class="p">)</span> <span class="n">var</span> <span class="n">DB</span> <span class="p">=</span> <span class="n">make</span><span class="p">(</span><span class="n">map</span><span class="p">[</span><span class="k">string</span><span class="p">]</span><span class="k">string</span><span class="p">)</span> <span class="n">func</span> <span class="n">setupRouter</span><span class="p">()</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Engine</span> <span class="p">{</span> <span class="p">//</span> <span class="n">Disable</span> <span class="n">Console</span> <span class="n">Color</span> <span class="p">//</span> <span class="n">gin</span><span class="p">.</span><span class="n">DisableConsoleColor</span><span class="p">()</span> <span class="n">r</span> <span class="p">:=</span> <span class="n">gin</span><span class="p">.</span><span class="n">Default</span><span class="p">()</span> <span class="p">//</span> <span class="n">Get</span> <span class="err">请求</span> <span class="n">r</span><span class="p">.</span><span class="n">GET</span><span class="p">(</span><span class="s2">"/ping"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="k">String</span><span class="p">(</span><span class="m">200</span><span class="p">,</span> <span class="s2">"pong"</span><span class="p">)</span> <span class="p">//</span> <span class="err">返回字符串</span> <span class="p">})</span> <span class="p">//</span> <span class="n">Get</span> <span class="n">user</span> <span class="n">value</span> <span class="n">r</span><span class="p">.</span><span class="n">GET</span><span class="p">(</span><span class="s2">"/user/:name"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">user</span> <span class="p">:=</span> <span class="n">c</span><span class="p">.</span><span class="n">Params</span><span class="p">.</span><span class="n">ByName</span><span class="p">(</span><span class="s2">"name"</span><span class="p">)</span> <span class="p">//</span> <span class="err">获取参数</span> <span class="n">value</span><span class="p">,</span> <span class="n">ok</span> <span class="p">:=</span> <span class="n">DB</span><span class="p">[</span><span class="n">user</span><span class="p">]</span> <span class="k">if</span> <span class="n">ok</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span><span class="m">200</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span><span class="s2">"user"</span><span class="p">:</span> <span class="n">user</span><span class="p">,</span> <span class="s2">"value"</span><span class="p">:</span> <span class="n">value</span><span class="p">})</span> <span class="p">//</span> <span class="err">返回</span> <span class="n">JSON</span> <span class="err">数据</span> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span><span class="m">200</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span><span class="s2">"user"</span><span class="p">:</span> <span class="n">user</span><span class="p">,</span> <span class="s2">"status"</span><span class="p">:</span> <span class="s2">"no value"</span><span class="p">})</span> <span class="p">}</span> <span class="p">})</span> <span class="p">//</span> <span class="n">Authorized</span> <span class="n">group</span> <span class="p">(</span><span class="k">uses</span> <span class="n">gin</span><span class="p">.</span><span class="n">BasicAuth</span><span class="p">()</span> <span class="n">middleware</span><span class="p">)</span> <span class="p">//</span> <span class="n">Same</span> <span class="n">than</span><span class="p">:</span> <span class="p">//</span> <span class="n">authorized</span> <span class="p">:=</span> <span class="n">r</span><span class="p">.</span><span class="n">Group</span><span class="p">(</span><span class="s2">"/"</span><span class="p">)</span> <span class="p">//</span> <span class="n">authorized</span><span class="p">.</span><span class="n">Use</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">BasicAuth</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">Credentials</span><span class="p">{</span> <span class="p">//</span> <span class="s2">"foo"</span><span class="p">:</span> <span class="s2">"bar"</span><span class="p">,</span> <span class="p">//</span> <span class="s2">"manu"</span><span class="p">:</span> <span class="s2">"123"</span><span class="p">,</span> <span class="p">//}))</span> <span class="n">authorized</span> <span class="p">:=</span> <span class="n">r</span><span class="p">.</span><span class="n">Group</span><span class="p">(</span><span class="s2">"/"</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">BasicAuth</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">Accounts</span><span class="p">{</span> <span class="s2">"foo"</span><span class="p">:</span> <span class="s2">"bar"</span><span class="p">,</span> <span class="p">//</span> <span class="n">user</span><span class="p">:</span><span class="n">foo</span> <span class="n">password</span><span class="p">:</span><span class="n">bar</span> <span class="s2">"manu"</span><span class="p">:</span> <span class="s2">"123"</span><span class="p">,</span> <span class="p">//</span> <span class="n">user</span><span class="p">:</span><span class="n">manu</span> <span class="n">password</span><span class="p">:</span><span class="m">123</span> <span class="p">}))</span> <span class="p">//</span> <span class="err">认证检查</span> <span class="n">authorized</span><span class="p">.</span><span class="n">POST</span><span class="p">(</span><span class="s2">"admin"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">user</span> <span class="p">:=</span> <span class="n">c</span><span class="p">.</span><span class="n">MustGet</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">AuthUserKey</span><span class="p">).(</span><span class="k">string</span><span class="p">)</span> <span class="p">//</span> <span class="n">Parse</span> <span class="n">JSON</span> <span class="n">var</span> <span class="n">json</span> <span class="n">struct</span> <span class="p">{</span> <span class="n">Value</span> <span class="k">string</span> <span class="p">`</span><span class="n">json</span><span class="p">:</span><span class="s2">"value"</span> <span class="n">binding</span><span class="p">:</span><span class="s2">"required"</span><span class="p">`</span> <span class="p">}</span> <span class="k">if</span> <span class="n">c</span><span class="p">.</span><span class="n">Bind</span><span class="p">(&amp;</span><span class="n">json</span><span class="p">)</span> <span class="p">==</span> <span class="n">nil</span> <span class="p">{</span> <span class="n">DB</span><span class="p">[</span><span class="n">user</span><span class="p">]</span> <span class="p">=</span> <span class="n">json</span><span class="p">.</span><span class="n">Value</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span><span class="m">200</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span><span class="s2">"status"</span><span class="p">:</span> <span class="s2">"ok"</span><span class="p">})</span> <span class="p">}</span> <span class="p">})</span> <span class="n">return</span> <span class="n">r</span> <span class="p">}</span> <span class="n">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="n">r</span> <span class="p">:=</span> <span class="n">setupRouter</span><span class="p">()</span> <span class="p">//</span> <span class="n">Listen</span> <span class="k">and</span> <span class="n">Server</span> <span class="k">in</span> <span class="m">0.0.0.0</span><span class="p">:</span><span class="m">8080</span> <span class="n">r</span><span class="p">.</span><span class="nf">Run</span><span class="p">(</span><span class="s2">":9095"</span><span class="p">)</span> <span class="p">}</span> </code></pre></div></div> <h3 id="初探">初探</h3> <p>看上面的例子,你几乎只需要定义好 path, handlefunc 即可:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>type HandlerFunc func(*Context) </code></pre></div></div> <p>gin 实现了 servehttp 的方法:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// Conforms to the http.Handler interface. func (engine *Engine) ServeHTTP(w http.ResponseWriter, req *http.Request) { c := engine.pool.Get().(*Context) c.writermem.reset(w) c.Request = req c.reset() engine.handleHTTPRequest(c) engine.pool.Put(c) } </code></pre></div></div> <p>详细的 path -&gt; handle 的逻辑这里不详细展开.</p> <h3 id="参数获取">参数获取</h3> <p>获取 path 中变量:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>router.GET("/user/:name", func(c *gin.Context) { name := c.Param("name") c.String(http.StatusOK, "Hello %s", name) }) </code></pre></div></div> <p>获取URL中参数:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> // The request responds to a url matching: /welcome?firstname=Jane&amp;lastname=Doe router.GET("/welcome", func(c *gin.Context) { firstname := c.DefaultQuery("firstname", "Guest") lastname := c.Query("lastname") // shortcut for c.Request.URL.Query().Get("lastname") c.String(http.StatusOK, "Hello %s %s", firstname, lastname) }) </code></pre></div></div> <p>获取 POST 请求参数:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>//POST /post?id=1234&amp;page=1 HTTP/1.1 //Content-Type: application/x-www-form-urlencoded // //name=manu&amp;message=this_is_great router.POST("/form_post", func(c *gin.Context) { message := c.PostForm("message") nick := c.DefaultPostForm("nick", "anonymous") c.JSON(200, gin.H{ "status": "posted", "message": message, "nick": nick, }) }) </code></pre></div></div> <p>获取上传文件:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>file, _ := c.FormFile("file") //or form, _ := c.MultipartForm() files := form.File["upload[]"] </code></pre></div></div> <h3 id="返回">返回</h3> <p>HTML :</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>router.LoadHTMLGlob("templates/*") c.HTML(http.StatusOK, "index.tmpl", gin.H{ "title": "Main website", }) </code></pre></div></div> <p>JSON 格式:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c.JSON(http.StatusOK, gin.H{"message": "hey", "status": http.StatusOK}) var msg struct { Name string `json:"user"` Message string Number int } msg.Name = "Lena" msg.Message = "hey" msg.Number = 123 // Note that msg.Name becomes "user" in the JSON // Will output : {"user": "Lena", "Message": "hey", "Number": 123} c.JSON(http.StatusOK, msg) </code></pre></div></div> <p>XML:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c.XML(http.StatusOK, gin.H{"message": "hey", "status": http.StatusOK}) </code></pre></div></div> <p>静态文件:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>router.Static("/assets", "./assets") </code></pre></div></div> <h3 id="状态码">状态码</h3> <p>重定向:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c.Redirect(http.StatusMovedPermanently, "http://www.google.com/") </code></pre></div></div> <h3 id="使用中间件">使用中间件</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>func (engine *Engine) Use(middleware ...HandlerFunc) IRoutes { engine.RouterGroup.Use(middleware...) engine.rebuild404Handlers() engine.rebuild405Handlers() return engine } </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>r.Use(gin.Recovery()) </code></pre></div></div> <h3 id="注意">注意</h3> <blockquote> <p>When starting new Goroutines inside a middleware or handler, you SHOULD NOT use the original context inside it, you have to use a read-only copy.</p> </blockquote> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>r.GET("/long_async", func(c *gin.Context) { // create copy to be used inside the goroutine cCp := c.Copy() go func() { // simulate a long task with time.Sleep(). 5 seconds time.Sleep(5 * time.Second) // note that you are using the copied context "cCp", IMPORTANT log.Println("Done! in path " + cCp.Request.URL.Path) }() }) </code></pre></div></div> <h3 id="多服务">多服务</h3> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">main</span> <span class="n">import</span> <span class="p">(</span> <span class="s2">"log"</span> <span class="s2">"net/http"</span> <span class="s2">"time"</span> <span class="s2">"github.com/gin-gonic/gin"</span> <span class="s2">"golang.org/x/sync/errgroup"</span> <span class="p">)</span> <span class="n">var</span> <span class="p">(</span> <span class="n">g</span> <span class="n">errgroup</span><span class="p">.</span><span class="n">Group</span> <span class="p">)</span> <span class="n">func</span> <span class="n">router01</span><span class="p">()</span> <span class="n">http</span><span class="p">.</span><span class="n">Handler</span> <span class="p">{</span> <span class="n">e</span> <span class="p">:=</span> <span class="n">gin</span><span class="p">.</span><span class="n">New</span><span class="p">()</span> <span class="n">e</span><span class="p">.</span><span class="n">Use</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">Recovery</span><span class="p">())</span> <span class="n">e</span><span class="p">.</span><span class="n">GET</span><span class="p">(</span><span class="s2">"/"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span> <span class="n">http</span><span class="p">.</span><span class="n">StatusOK</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span> <span class="s2">"code"</span><span class="p">:</span> <span class="n">http</span><span class="p">.</span><span class="n">StatusOK</span><span class="p">,</span> <span class="s2">"error"</span><span class="p">:</span> <span class="s2">"Welcome server 01"</span><span class="p">,</span> <span class="p">},</span> <span class="p">)</span> <span class="p">})</span> <span class="n">return</span> <span class="n">e</span> <span class="p">}</span> <span class="n">func</span> <span class="n">router02</span><span class="p">()</span> <span class="n">http</span><span class="p">.</span><span class="n">Handler</span> <span class="p">{</span> <span class="n">e</span> <span class="p">:=</span> <span class="n">gin</span><span class="p">.</span><span class="n">New</span><span class="p">()</span> <span class="n">e</span><span class="p">.</span><span class="n">Use</span><span class="p">(</span><span class="n">gin</span><span class="p">.</span><span class="n">Recovery</span><span class="p">())</span> <span class="n">e</span><span class="p">.</span><span class="n">GET</span><span class="p">(</span><span class="s2">"/"</span><span class="p">,</span> <span class="n">func</span><span class="p">(</span><span class="n">c</span> <span class="p">*</span><span class="n">gin</span><span class="p">.</span><span class="n">Context</span><span class="p">)</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="n">JSON</span><span class="p">(</span> <span class="n">http</span><span class="p">.</span><span class="n">StatusOK</span><span class="p">,</span> <span class="n">gin</span><span class="p">.</span><span class="n">H</span><span class="p">{</span> <span class="s2">"code"</span><span class="p">:</span> <span class="n">http</span><span class="p">.</span><span class="n">StatusOK</span><span class="p">,</span> <span class="s2">"error"</span><span class="p">:</span> <span class="s2">"Welcome server 02"</span><span class="p">,</span> <span class="p">},</span> <span class="p">)</span> <span class="p">})</span> <span class="n">return</span> <span class="n">e</span> <span class="p">}</span> <span class="n">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="n">server01</span> <span class="p">:=</span> <span class="p">&amp;</span><span class="n">http</span><span class="p">.</span><span class="n">Server</span><span class="p">{</span> <span class="n">Addr</span><span class="p">:</span> <span class="s2">":8080"</span><span class="p">,</span> <span class="n">Handler</span><span class="p">:</span> <span class="n">router01</span><span class="p">(),</span> <span class="n">ReadTimeout</span><span class="p">:</span> <span class="m">5</span> <span class="p">*</span> <span class="n">time</span><span class="p">.</span><span class="n">Second</span><span class="p">,</span> <span class="n">WriteTimeout</span><span class="p">:</span> <span class="m">10</span> <span class="p">*</span> <span class="n">time</span><span class="p">.</span><span class="n">Second</span><span class="p">,</span> <span class="p">}</span> <span class="n">server02</span> <span class="p">:=</span> <span class="p">&amp;</span><span class="n">http</span><span class="p">.</span><span class="n">Server</span><span class="p">{</span> <span class="n">Addr</span><span class="p">:</span> <span class="s2">":8081"</span><span class="p">,</span> <span class="n">Handler</span><span class="p">:</span> <span class="n">router02</span><span class="p">(),</span> <span class="n">ReadTimeout</span><span class="p">:</span> <span class="m">5</span> <span class="p">*</span> <span class="n">time</span><span class="p">.</span><span class="n">Second</span><span class="p">,</span> <span class="n">WriteTimeout</span><span class="p">:</span> <span class="m">10</span> <span class="p">*</span> <span class="n">time</span><span class="p">.</span><span class="n">Second</span><span class="p">,</span> <span class="p">}</span> <span class="n">g</span><span class="p">.</span><span class="n">Go</span><span class="p">(</span><span class="n">func</span><span class="p">()</span> <span class="n">error</span> <span class="p">{</span> <span class="n">return</span> <span class="n">server01</span><span class="p">.</span><span class="n">ListenAndServe</span><span class="p">()</span> <span class="p">})</span> <span class="n">g</span><span class="p">.</span><span class="n">Go</span><span class="p">(</span><span class="n">func</span><span class="p">()</span> <span class="n">error</span> <span class="p">{</span> <span class="n">return</span> <span class="n">server02</span><span class="p">.</span><span class="n">ListenAndServe</span><span class="p">()</span> <span class="p">})</span> <span class="k">if</span> <span class="n">err</span> <span class="p">:=</span> <span class="n">g</span><span class="p">.</span><span class="nf">Wait</span><span class="p">();</span> <span class="n">err</span> <span class="c1">!= nil { </span> <span class="nb">log</span><span class="p">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>未完待续</p> Jmeter 介绍 2018-04-05T00:00:00+08:00 /2018/04/05/jmeter-usermanual <h3 id="前言">前言</h3> <p>在做压力测试前,先思考一下:</p> <ul> <li>预期平均用户数是多少?</li> <li>预期高峰用户数是多少?</li> </ul> <h3 id="概念">概念</h3> <h4 id="test-plan-测试计划">Test Plan (测试计划)</h4> <p>测试计划,即 Jmeter 的执行内容,完整的测试计划包括: Thread Groups(线程组), logic controllers(逻辑控制器), sample generating controllers(取样控制器), listeners(监听器), timers(定时器), assertions(断言), configuration elements(配置元素).</p> <p>一个测试计划,至少要包含: 一个线程组,一个或多个取样器。</p> <h4 id="thread-groups线程组">Thread Groups(线程组)</h4> <p>测试计划的入口。用于设置, 线程数,递增线程时间、总的运行次数与时间(持续时间)。</p> <p>eg:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Test Plan Thread Group - Once Only Controller * Login Request (an HTTP Request) - Load Search Page (HTTP Sampler) - Interleave Controller * Search "A" (HTTP Sampler) * Search "B" (HTTP Sampler) * HTTP default request (Configuration Element) - HTTP default request (Configuration Element) - Cookie Manager (Configuration Element) </code></pre></div></div> <p>上面的测试线程组执行一次 only once controller, 然后加载 search page, 交替执行 search a/b(按顺序交替选择),执行顺序是:</p> <ul> <li>Configuration elements</li> <li>Pre-Processors</li> <li>Timers</li> <li>Sampler</li> <li>Post-Processors (unless SampleResult is null)</li> <li>Assertions (unless SampleResult is null)</li> <li>Listeners (unless SampleResult is null)</li> </ul> <h4 id="controllers-控制器">controllers( 控制器)</h4> <p>JMeter 有两种控制器:Samplers and Logical Controllersa 。 Samplers: 请求方式。(http, ftp, tcp etc)<br /> Logical controllers: 发送请求的逻辑,loop/while/once only etc.</p> <h4 id="listeners监听器">Listeners(监听器)</h4> <p>监听器用来对 JMeter test cases 的执行进行展示、汇总、分析。View Results Tree, Graph Results listeners,Aggregate Report 等等。</p> <h4 id="timers定时器">Timers(定时器)</h4> <p>用于增加请求间的延时。</p> <h4 id="assertions-断言">assertions (断言)</h4> <p>用来测试、标记请求是否成功</p> <h4 id="pre-processorpost-processor">Pre-Processor/Post-Processor</h4> <p>用于请求的前后置处理。比如使用 HTTP URL Re-writing Modifier 来重置URL。</p> <h3 id="分布式部署"><a href="https://jmeter.apache.org/usermanual/remote-test.html">分布式部署</a></h3> <p>我们使用一台机器来进行说明(实际上,最好每台机器部署一个slave 节点)。</p> <p>启动 server, 端口号 9091/9092/9093:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>SERVER_PORT=9091 /usr/local/Cellar/jmeter/3.2/libexec/bin/jmeter-server -Djava.rmi.server.hostname=localhost SERVER_PORT=9092 /usr/local/Cellar/jmeter/3.2/libexec/bin/jmeter-server -Djava.rmi.server.hostname=localhost SERVER_PORT=9093 /usr/local/Cellar/jmeter/3.2/libexec/bin/jmeter-server -Djava.rmi.server.hostname=localhost </code></pre></div></div> <p>使用 GUI 方式,启动 client or master,</p> <p>vi /usr/local/Cellar/jmeter/3.2/libexec/bin/jmeter.properties, 修改 remote_hosts:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>remote_hosts=127.0.0.1:9091,127.0.0.1:9092,127.0.0.1:9093 </code></pre></div></div> <p>使用非 GUI 方式启动:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>jmeter -n -t ~/Downloads/URLRewritingExample.jmx -R127.0.0.1:9093,127.0.0.1:9091,127.0.0.1:9092 </code></pre></div></div> <h3 id="监控">监控</h3> <p>这里在 mac 下安装 influxdb 与 grafana 来配置<a href="https://jmeter.apache.org/usermanual/realtime-results.html">实时监控</a></p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install influxdb brew services start influxdb </code></pre></div></div> <p>influx -precision rfc3339</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CREATE DATABASE grafana CREATE DATABASE jmeter SHOW DATABASES; </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew install grafana brew services start grafana </code></pre></div></div> <p>打开: http://localhost:3000/ 登录设置 <br /> 添加 infuxdb 源。</p> <p>jmeter 测试时,添加 backend listeners:<code class="highlighter-rouge">http://127.0.0.1:8086/write?db=jmeter</code> 试试查看数据。可以在命令行中查看 influxdb 中的数据:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>SHOW MEASUREMENTS; SHOW TAG KEYS FROM "jmeter"; select * from jmeter </code></pre></div></div> <p>维度:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>show field keys time application avg count countError endedT hit max maxAT meanAT min minAT pct90.0 pct95.0 pct99.0 startedT statut transaction </code></pre></div></div> <h3 id="最佳实践">最佳实践</h3> <h4 id="定义变量">定义变量</h4> <p>User Defined Variables, 定义变量:<br /> eg: name: resources_folder, value: /, desc: folder path</p> <p>Define CSV DATA:<br /> name: loginData <br /> Variable Name(comma-delimited): login, password <br /> <a href="http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config">share mode</a>: All threads,所有线程组共享文件;Current thread group: 当前的线程组共享文件;Current thread: 线程共享,线程组内不共享。</p> <p>假如你有1万个用户的文件,想要每个线程的用户不同,则设置成Current thread group;如果可以相同,则可以设置成 Current thread.</p> <h4 id="减少资源使用">减少资源使用</h4> <ul> <li>使用非 GUI 模式: jmeter -n -t test.jmx -l test.jtl</li> <li>尽量少使用 Listeners, 不在压测是使用: “View Results Tree” or “View Results in Table”</li> <li>不实用 functional mode</li> <li>Use CSV output rather than XML</li> <li>只保存必须的数据</li> <li>尽量少使用断言</li> <li></li> </ul> <h3 id="提醒">提醒</h3> <p><code class="highlighter-rouge">Using GUI mode as described here should only be used when debugging your Test Plan. To run the real load test, use NON-GUI mode.</code></p> <p>压测千万别用 GUI 模式,一定要用 模式.</p> <h3 id="参考资料">参考资料</h3> <p><a href="https://www.ibm.com/developerworks/cn/opensource/os-pressiontest/">使用 JMeter 完成常用的压力测试</a> <br /> <a href="https://www.blazemeter.com/blog/how-to-use-grafana-to-monitor-jmeter-non-gui-results-part-2">How to Use Grafana to Monitor JMeter Non-GUI Results - Part 2</a></p>