[2018-11-29] 1 crawler/crawler_sys/framework/scrap_list_page_async.py 1.1 lst_page_conf.ini ת��crawler/crawler_sys/config/sites/list_page_urls.ini�� 1.2 list_page_urls.ini ��ÿ��siteһ��[section] header�����е�site����Ҫ��crawler/crawler_sys/framework/platform_crawler_register.py����һ�£� 1.3 args.platform default='' (������'��Ѷ��Ƶ'), ��parse arg��ʱ���жϣ����platform����=='' ֱ���˳��� 1.4 args.platform �����Ϊ�գ��ж��Ƿ���latform_crawler_register.py�������ǣ������˳��� 2 ���ƹ淶�������ļ����ͺ�������������ȼ������������ʱ���ٸģ� lst_page -> list_page [2018-12-25] 1 for releaser_page crawler, the name of function must be releaser_page so that we can import the function in framework 2 for the releaser_page function, the input variable is releaserUrl, other functions such as get_releaser_id and get_releaser_uk must be included in this function 3 es_index and doc_type must be given so that we can reduce some if/else in output process. At the beginning, if es_index is None, the es_index is default to crawler-data-raw