15:04:39 <yamahata> #startmeeting neutron_northbound
15:04:39 <odl_meetbot> Meeting started Mon Mar 13 15:04:39 2017 UTC.  The chair is yamahata. Information about MeetBot at http://ci.openstack.org/meetbot.html.
15:04:39 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:04:39 <odl_meetbot> The meeting name has been set to 'neutron_northbound'
15:04:47 <yamahata> #chair vthapar
15:04:47 <odl_meetbot> Current chairs: vthapar yamahata
15:04:56 <yamahata> #topic agenda bashing and roll call
15:05:16 <yamahata> anything else to discuss to addition to usual topic?
15:05:26 <yamahata> I have the unstability of openstack CI.
15:05:36 <yamahata> s/I/We/
15:05:53 <vthapar> #info vthapar
15:06:01 <yamahata> #info yamahata
15:06:03 <vthapar> just patches/bugs.
15:06:10 <yamahata> okay
15:06:17 <yamahata> #topic Announcements
15:06:32 <yamahata> openstack summit is approaching.
15:06:50 <yamahata> the presentation proposal related to Isaku were all rejected.
15:06:58 <yamahata> vthapar: How about you?
15:07:17 <vthapar> yamahata: rejected.
15:07:39 <yamahata> :-(.
15:08:01 <yamahata> anyway will you attend one of openstack summit or opendaylight developer design forum?
15:08:17 <vthapar> I'll be at DDF for sure.
15:08:23 <yamahata> That's good.
15:08:38 <yamahata> any other announcement?
15:09:43 <yamahata> seems nothing else. move on
15:09:45 <yamahata> #topic action items from last meeting
15:10:06 <yamahata> last week we have many action items. But not completed yet.
15:10:24 <yamahata> please file/update blueprints/specs.
15:11:13 <yamahata> #topic Carbon planning
15:11:39 <yamahata> regarding to odl neutron northbound, it's code freeze. So only bug fixes would be merged.
15:11:47 <yamahata> I suppose 1 or 2 ptaches left
15:12:05 <vthapar> what is the M5 date?
15:12:32 <yamahata> March 16. it's code freeze.
15:12:40 <yamahata> Right now it's api freeze.
15:13:02 <yamahata> So this week, pending patches will be merged.
15:13:23 <yamahata> No big one left.
15:14:00 <yamahata> I'm wondering abpout moving port:status patch to Nitrogen.
15:14:28 <yamahata> #topic Pike planning
15:14:32 <vthapar> yamahata: yeah, was about to mention. too short time to get it in good shape. we'll probaby want to backport it later.
15:14:43 <yamahata> #undo
15:14:43 <odl_meetbot> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x16afa10>
15:15:04 <vthapar> what about https://git.opendaylight.org/gerrit/#/c/51534/
15:15:30 <yamahata> I'd like to merge it.
15:15:40 <vthapar> this is the one with tenant-id change.
15:15:43 <yamahata> because it won't break the depenent projects.
15:16:06 <vthapar> yeah.
15:16:21 <vthapar> need to deprecate it now so can remove it in Nitrogen.
15:16:35 <yamahata> Yeah.
15:17:11 <vthapar> any more expected for hostconfig?
15:17:37 <yamahata> Only test and bug fix.
15:17:56 <yamahata> Probably multinode environment would need more tests.
15:18:24 <yamahata> scalability tests needs to be done. but it's for networking-odl side.
15:19:32 <yamahata> #topic Pike planning
15:19:42 <yamahata> we're filing blueprints/specs
15:19:58 <yamahata> anything to discuss here?
15:20:39 <vthapar> nothing right now from me, currently busy with existing patches/bugs and testing.
15:20:47 <yamahata> Cool.
15:20:51 <yamahata> #topic patches/bugs
15:21:04 <yamahata> Now openstack CI is very unstable and it's blocking some patches.
15:21:14 <yamahata> I'm wondering to disable some test cases for now.
15:21:46 <yamahata> With carbon + Ocata(or master), floating ip is still broken.
15:22:13 <yamahata> vthapar: any bugs/patches to discuss?
15:22:34 <vthapar> https://review.openstack.org/#/c/443018/ is high on priority. need it backported to newton too.
15:22:58 <vthapar> it needs https://review.openstack.org/443361 to go in first I think.
15:23:31 <vthapar> as you said, CI unstabality is dealying it. patch looks good.
15:23:40 <yamahata> stable/newton seems broken somehow too.
15:23:53 <yamahata> https://review.openstack.org/#/c/443361/ doesn't fix the unstability fully.
15:24:52 <yamahata> For master branch, I'm going to skip some tests to unblock.
15:25:34 <reedip> #info reedip
15:25:36 <vthapar> +1
15:25:42 <yamahata> For ocata, newton, we need more? or to wait for carbon netvirt to fix?
15:25:48 <reedip> I thought the meeting was after an hour ??
15:25:50 <reedip> :)
15:26:02 <vthapar> reedip: Daylight Savings.
15:26:07 <yamahata> reedip: hello.
15:26:11 <reedip> I thought so
15:26:29 <yamahata> right now we're discussing bugs/patches.
15:26:41 <yamahata> recent unstability of CI.
15:26:52 <reedip> yeah, reading logs
15:27:25 <yamahata> any other buts/patches to discuss?
15:27:36 <yamahata> s/buts/bugs
15:28:26 <vthapar> I think rest we can take up once CI is stable enough. no point adding more patches
15:28:41 <vthapar> when CSIT is not stable.
15:28:57 <yamahata> makes sense
15:29:06 <yamahata> next topic
15:29:07 <yamahata> #topic open mike
15:29:12 <yamahata> anything else
15:29:14 <yamahata> ?
15:29:51 <vthapar> wanted to bring up one problem we face when trying to debug tempest failures in Netvirt.
15:30:00 <yamahata> For pike planning we have some of blueprints.and one spec uploaded.
15:30:09 <yamahata> vthapar: please go ahead.
15:30:22 <vthapar> for netvirt test suites we capture some dumps at end of every test that helps debug
15:30:34 <vthapar> but with tempest, the cleanup at end of every test makes it impossible to do so.
15:30:47 <vthapar> can we do something about it?
15:31:14 <yamahata> By dumps, what do you mean?
15:31:27 <vthapar> the dumps I mentioned are mainly MDSAL dumps, OVS-vsctl and ovs-ofctl dumps.
15:31:40 <yamahata> Oh, I see. those would be very helpful.
15:32:04 <yamahata> For now openstack CI, those are taken only befire/after all the tests.
15:32:22 <vthapar> we check for ports created, flows and groups configured and also the different MDSAL to check if internal logic set those up correctly.
15:32:34 <yamahata> It's worthwhile to look into it.
15:32:37 <vthapar> yes, that is why debugging individual test failure becomes difficult.
15:32:59 <vthapar> even if we can do it on a custom job in sandbox, would be helpful.
15:33:02 <yamahata> Ideally tempest wouldprovide such hooks.
15:33:18 <yamahata> At worst monkey patching would be resorted.
15:34:43 <yamahata> can you please file a rfe bug to track it?
15:34:43 <reedip> why dont u execute individual tempest test , if you are facing an issue which is easily reproducible
15:35:26 <vthapar> reedip: that is an option we're exploring, but mostly coz it only happens because of sequence of tests.
15:36:05 <reedip> oh, so you think a set of tests are failing ? How about using pdb
15:36:22 <vthapar> previous tests and their cleanups trigger activity on ODL side, so it is mostly timing issue.
15:36:32 <reedip> I mean you can isolate a larger set of test cases and use simple pdb if its not time constrained?
15:36:42 <reedip> ok, pdb wont work
15:36:46 <reedip> :)
15:36:49 <vthapar> yep :)
15:36:58 <reedip> yeah, you need dumps then
15:37:22 <vthapar> some cases it could be as simple as ODL took longer than tempest waits for it, but we can't know.
15:37:45 <reedip> interesting
15:37:53 <vthapar> we had quite a few issues in ODL tests, where test failed but dumps when captured showed correct information, meaning data was in-flight when test last checkd.
15:38:07 <vthapar> in such cases we increase timeout or add more retries.
15:38:31 <vthapar> and sometimes it can be as simple as previous test not cleaning up correctly.
15:38:41 <vthapar> today we can't rule out such issues.
15:39:26 <reedip> instead of clearing and cleaning up the test, can we add a decorator which checks the ODL state ?
15:39:29 <yamahata> currently we're running tempest test serially because running in parallel causes faulres.
15:39:35 <reedip> before going up for cleanup
15:39:37 <reedip> in tempest ?
15:40:09 <vthapar> reedip: that is what we're discussing, what are the options we have.
15:40:25 <reedip> yamahata : I think thats one of the items rajivk__ is looking into
15:40:32 <vthapar> as per yamahata there should be hooks in tempest to allow us to do that.
15:40:49 <yamahata> I guess so. tempest or testrepository.
15:41:00 <reedip> vthapar : I second that thought. We can add our own functions to check ODL status if tempest doesnt give us a hook
15:43:54 <yamahata> Or we can propose enhancement to tempest at worst.
15:44:44 <reedip> yamahata : worth a try though I dont think project specific patches would be welcome, but we can always try :)
15:45:33 <yamahata> I mean, adding a general hooks so that any project can utilise it.
15:45:39 <vthapar> reedip: not project specific, but generic pre_test/post_test hooks that any poject can use.
15:45:44 <yamahata> not odl specific one.
15:45:44 <rajivk__> reedip, may be we can ask to provide some generic feature.
15:46:08 <reedip> yamahata , vthapar : I think pre-post test functions would be there
15:46:18 <reedip> rajivk__ ^^
15:46:29 <rajivk__> And then we hook our intended function or method.
15:49:40 <yamahata> Hmm sounds great
15:49:50 <reedip> yamahata , vthapar : https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
15:49:58 <reedip> It mentions something about pre-test-hook
15:50:35 <reedip> you can also use a custom devstack plugin ( another option )
15:50:49 <reedip> I think we can discuss this on the ML , and continue with other topics for now ???
15:51:17 <vthapar> +1 to discussing on ML.
15:51:37 <rajivk__> +1 ML
15:51:38 <yamahata> Do you mean function pre_test_hook {?
15:51:42 <yamahata> +1 ML.
15:52:06 <yamahata> that pre_test_hook is for hook before all tempest tests.
15:52:20 <yamahata> not for each testcases.
15:52:35 <yamahata> okay now we have 8 mins left.
15:52:48 <manjeets> sorry I was on another meeting
15:53:00 <yamahata> vthapar: can you please file a rft bug?
15:53:01 <manjeets> yamahata, i sent a patch last friday for updating conf
15:53:09 <yamahata> Cool. which one?
15:53:13 <rajivk__> last meeting discussion is pending
15:53:31 <manjeets> https://review.openstack.org/#/c/444546/
15:53:37 <vthapar> #action vthapar to file RFE bug for pre/post test hooks for ODL in tempest
15:53:47 <rajivk__> yamahata, mkolesni, https://review.openstack.org/#/c/407784/
15:54:12 <yamahata> regarding to 407784, today we don't have mike.
15:54:46 <yamahata> I'd like to monitor the backlog size of journal and average processing time.
15:55:01 <yamahata> Especially with openstack CI of tempest and rally.
15:55:15 <yamahata> As long as it's realised, I don't care of its implementation.
15:55:59 <rajivk__> yamahata, this patch is pending since 2 weeks. We have to agree on a implentation.
15:56:15 <yamahata> I suspect the reason why Mike is against it is, he's planning to refactor journal dependency.
15:56:18 <rajivk__> May be we can discuss it, on IRC
15:57:06 <rajivk__> Even after refactoring, we need to monitor, right?
15:57:27 <yamahata> Yes.
15:58:03 <yamahata> My take is the current implementation way wouldn't affect his plan.
15:58:08 <yamahata> But he doesn't seem to think so.
15:58:32 <rajivk__> hmm, we have bring all parties for discussion together.
15:58:55 <rajivk__> We don;t discuss during week and in weekly meetings we don't get time.
15:59:19 <yamahata> Okay.
15:59:27 <rajivk__> Can we plan a time for a discussion over IRC>
15:59:33 <yamahata> Sure.
15:59:44 <yamahata> then let me send a mail on it.
15:59:51 <yamahata> for plan irc.
15:59:53 <rajivk__> yamahata, thanks
16:00:12 <yamahata> #action yamahata send a mail for planning IRC discussion on https://review.openstack.org/#/c/407784/
16:00:22 <yamahata> anything else discuss?
16:00:32 <yamahata> we're running out of time.
16:01:07 <yamahata> seems nothing else.
16:01:11 <yamahata> thank you everyone
16:01:31 <yamahata> #topic cookies
16:01:36 <yamahata> #endmeeting